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Executive Summary 


Introduction and Context 

California’s dynamic economy depends on having a large and skilled workforce; consequently, the state 
must continually support and refine efforts to provide workers with employer-valued competencies. 

Given the wide range of regional and state needs across this vast state, ensuring that the workforce 
has the training to keep up with labor market demands is difficult. The California Community Colleges’ 
(CCC) Economic and Workforce Development Program (EWD) aims to support the development 
of a workforce that will promote California’s economic development by connecting employers and 
educators. This report summarizes the findings of an independent evaluation conducted on EWD. 

EWD aims to “invest in California’s economic growth and global competitiveness through 
industry-specific education, training, and services that contribute to a highly-skilled and 
productive workforce.” To support its mission, EWD created a new structure focused on 
regional and sector service delivery. Each region selected five sectors— three priority and two 
emergent— based on their rapid growth, high demand for workers, and well-paying jobs. 

To provide services, EWD employs: 

• Sector navigators (SNs) to serve as statewide experts in their sector 

• Deputy sector navigators (DSNs) to address a given sector’s unique regional needs 

• Industry-driven regional collaboratives (IDRCs) to deliver short-term 
projects devoted to specific regional workforce needs 

• Technical assistance providers (TAPs) to provide support to the SNs and DSNs, 
including the Centers of Excellence for Labor Market Research (CoE) and 
LaunchBoard (a data system that tracks student progress and outcomes) 

This is the culminating report of the legislatively-mandated independent evaluation of EWD. The evaluation 
focused on three main components: the processes by which grantees operated (roles, coordination, 
funding, data use); the services offered; and the outcomes. Our review of these components are based 
on our perceptions. We were not able to observe activities, nor did we have access to data that would 
allow us to estimate outcomes. From July to December 2016, we analyzed quantitative data and grantee 
accountability reports provided by the California Community Colleges Chancellor’s Office (CCCCO), 
conducted 46 grantee interviews, and administered surveys to 1 1 0 participants comprised of grantees, 
career technical education (CTE) deans, regional consortia chairs, and employers. While we were able to 
gather rich information from EWD grantees, including SNs, DSNs, IDRCs, and TAPs, we faced significant 
barriers in other aspects of the evaluation. The quick timeline of this work limited our capacity to gather 
information from stakeholders. The design of EWD lacked a comparison group, and our inability to access 
relevant data forced us to make many concessions in reporting outcomes and to be unable to make 
causal claims about the program’s effectiveness. The subsections below summarize our evaluation’s main 
findings, EWD’s strengths and areas of challenge, and recommendations for program improvement. 
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EWD Design was Well Received as 
a Means of Structuring and Unifying 
Activities 

Overall, the evaluation found that EWD grantees 
provided the services and performed the 
responsibilities as described by their roles within 
EWD’s structure, and the roles and responsibilities 
were viewed favorably. DSNs focused primarily on 
connecting employers and community colleges 
and assisting in aligning curriculum; SNs served as 
experts in their sectors’ needs; CoEs supplied labor 
market information to grantees; and LaunchBoard 
provided data in support of accountability and 
program improvement. EWD grantees coordinated 
intentionally and frequently among themselves 
and with non-EWD stakeholders. The grantees 
braided together various funding sources in 
support of economic and workforce development, 
and they used data to understand market 
trends and basic demographics and to decide 
whether to revise programs or add new ones. 

EWD grantees provided a variety of services and 
activities in support of EWD goals and objectives. 
Grantees reported that they made 14 different types 
of services available to colleges and employers 
(see Tables 2 and 3 for a breakdown of services 
provided by EWD grantees). Some services, 
although similar, varied depending whether the 
target population was colleges or employers. 

For example, the professional development 
services provided by grantees were different for 
colleges than they were for employers. While 
colleges might receive a program on how to 
use new lab equipment, for example, employers 
might receive help with employee recruitment or 
with conducting a business needs assessment. 

In total, EWD grantees reported that they: 

• Provided 2,577 services to 
colleges and businesses. 

• Served 13,723 businesses, 41,252 
students, and 38,486 employees. 

These services were comprised of: 

• 32,935 hours of workshops and trainings. 

• 1 1 6,906 hours of contract education. 


• 64,866 hours of performance 
improvement training. 

• 115,146 hours of credit/non-credit instruction. 

Interviewees Said Students Gained Work 
Experience and had Promising Academic 
Outcomes 

In general, respondents reported that 
EWD improved employment outcomes for 
program participants. Grantees reported 
that EWD-funded programs placed: 

• 1,135 students or trainees in jobs. 

• 2,650 students in work-based 
learning programs, such as 
apprenticeships or internships. 

Students’ academic outcomes were promising, 
with students’ course-passing rates 1 increasing 
over the life of the grant in courses identified by 
the CCCCO as being EWD-related. There also was 
an increase in the number of both CTE- and EWD- 
related associate degrees and certificates earned. 
Given that these were simultaneous increases, it is 
not clear if students in EWD courses did the same 
as those in CTE courses generally, or if, since the 
efforts were braided, the impact was shared. 

Employers Stated that They Received 
Benefits from EWD Services 

Employers were also recipients of EWD 
services and reported the program was 
helpful. Grantees indicated in their quarterly 
reports that their services led to: 

• Hiring 1 ,498 employees. 

• Retaining 7,1 94 employees (i.e. 
reducing employee turnover). 

• Generating $75,430,844 in revenue. 

• Achieving $99,319,944 in sales. 

• Developing 75 new products. 

• Creating 94 new services. 

Employers surveyed noted that employers had 
limited awareness of EWD. Those who were 
familiar with EWD and those who were not both 
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reported a lack of knowledge about available 
EWD programs and services. Further signaling 
a need to build program awareness was the fact 
that employers indicated a need for services 
similar to the ones EWD already provided. 

Grantees and Stakeholders Believed 
the Program was Generally to 
Somewhat Effective in Advancing 
California’s Economy 

EWD grantees and stakeholders alike were generally 
positive about the effects they perceived EWD 
was having on California’s community colleges, 
employers, and workforce system. We found that: 

• Nearly all (97 percent) of DSNs believed 
the EWD program was meeting its goal of 
advancing California's economic growth and 
global competitiveness at least “somewhat 
well.” (47 percent reported “very well” and 
50 percent reported “somewhat well”). 

• More than half (61 percent) of CTE deans 
reported that EWD had been at least 
“somewhat effective” in fulfilling its mission. 

(1 7 percent reported “very effective” and 44 
percent reported “somewhat effective”). 

• All (1 00 percent) of the regional consortia 
chairs said EWD had been at least 
“somewhat effective” in fulfilling its mission. 

(29 percent reported “very effective” and 71 
percent reported “somewhat effective”). 

• All (1 00 percent) of the employers that knew 
of EWD believed the program was meeting its 
goal of advancing California’s economic growth 
and global competitiveness “somewhat well.” 

EWD’s Strengths and Areas of Challenge 

As the data above indicate, while there was 
general support for EWD, there also were some 
concerns; respondents provided information about 
both EWD strengths and challenges. Most of the 
perceived strengths stemmed from EWD’s design. 
Interviewees believed the program’s challenges 
were likely due to the revision of EWD structure and 
may be minimized as the new structure matures. 

The strengths were: 

• EWD’s ability to respond more quickly 
to employers’ dynamic needs, which 


improved employer engagement 
with the community colleges. 

• The Doing What Matters for Jobs and the 
Economy (DWM) framework helped connect 
programs with shared goals, promoted a 
mindset of care and service, and increased 
grantees’ focus on the community colleges. 

• The sense of camaraderie fostered by 
EWD among grantees and stakeholders 
around efforts to build regional 
economies and workforces. 

• The explicit focus on high growth sectors 
that allowed grantees to target and better 
understand the unique job skills of specific 
sectors and to concentrate on those that 
would reap greater returns for students, 
workers, and regional economies. 

• The building of career pathways starting 
in K-12 that help make workforce and 
economic development options more visible 
to students, teachers, and administrators. 

• The focus on continuous improvement, an 
effort that will promote EWD’s impact on 
economic and workforce development. 

The challenges were: 

• Lack of clarity for both DSNs and the 
stakeholders they work with about 
DSN roles and accountability. 

• Complicated and poorly functioning funding 
and grant management mechanisms. This 
led to grantee frustration and time spent 
figuring out contract issues rather than 
providing services to support EWD’s mission. 

• Data limitations and lack of clarity on 
metrics. As a result, grantees struggled 
to understand what services should be 
provided and how EWD was performing. 

• Lack of awareness about EWD activities 
and services by employers and community 
colleges, which affected grantees’ 

ability to connect with employers and 
coordinate with stakeholders. 

• Difficulty experienced by grantees in 
connecting with community colleges in 
order to meet EWD objectives. Grantees 
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cited CCCs’ rigid structure and the 
colleges’ lack of understanding about 
EWD and its lines of accountability. 

• Regional boundaries, which placed some 
DSNs in such large regions that they couldn’t 
consistently meet regional needs and 
prevented grantees from easily coordinating 
with stakeholder organizations in other 
regions. Additionally, DSNs wound up 

also serving adjacent regions that lacked 
DSNs with expertise in specific sectors. 

• Lack of clarity about how regions select 
priority and emergent sectors and, as a result, 
a lack of EWD buy-in by stakeholders. 

Recommendations for EWD Program 
Improvement 

Across all participants we interviewed and 
surveyed, stakeholders generally believed that 
EWD has succeeded in contributing to a skilled 
and productive workforce. The structure of the 
program under the Doing What Matters framework 
is relatively new, and this is reflected in participants’ 
statements about the program’s effectiveness, 
impact, strengths, and weaknesses. While some of 
the issues highlighted will likely work themselves 
out as the program matures, especially given 
the ongoing work to address recommendations 
from the CCC Board of Governors’ (BOG) Task 
Force on Workforce, Job Creation and a Strong 
Economy, we have identified areas that should 
be addressed to improve EWD’s success. 

Many of our recommendations overlap with 
those of the BOG task force and include: 

• Clarify grantee roles and responsibilities, 
while maintaining a transparent and 
consistent system of accountability. 

• Refine funding and grant management 
to improve EWD efficiency. 

• Improve data and metrics used for EWD 
improvement and accountability. 

• Build awareness of EWD, particularly for 
employers and community colleges. 

• Increase community colleges’ ability to 
respond quickly to employer needs. 


• Refine the regional focus to 
encourage greater collaboration. 

• Formalize a structure for sector 
selection that promotes a thoughtful 
and data-driven approach. 

Conclusion 

EWD is an important component of the 
state’s career technical education, or CTE, program. 
Using a multi-level delivery and support system 
and a regional and sector-specific structure, EWD 
works within the broader CTE-focused effort of 
DWM, connecting employers with educators to 
promote alignment of workforce development 
needs and community college educational 
offerings. EWD has experienced pains typical of 
a newly restructured program, such as difficulty 
with clarity in grantee roles, management of 
grants, development of a new data system, 
and program awareness and marketing. 

Most of the recommendations for EWD 
improvement largely reinforce previous ones 
regarding CTE, including our own at Edlnsights 2 
and those from the BOG task force. The CCCCO 
began acting on the BOG recommendations, and 
we expect many of ours will be addressed in this 
process. However, some of the recommendations 
require support from the state legislature and from 
the community colleges, not just from the CCCCO. 

Finally, a better understanding of the impact 
of EWD and other CCC programs depends on 
better data. The unveiling of LaunchBoard 2.0, 
targeted for 2017, could present opportunities for 
answering key questions regarding EWD outcomes 
and impact. Supporting further research would 
allow the state, the CCCCO, and the CCC system 
to make more informed decisions about which 
investments to target; what practices have the best 
outcomes, biggest impacts and greatest value- 
adds; and where outcomes and impact are less 
than is expected or demanded by stakeholders. 
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Introduction and Context: California 
Aims to Meet Workforce Needs 


California’s dynamic economy depends on having 
a large and skilled workforce; consequently, 
the state must continually support efforts to 
develop new workers with up-to-date skill sets 
(“skilling”), retrain incumbent workers for shifts in 
careers (“reskilling”), and build further expertise 
in incumbent workers (“upskilling”). Ensuring 
the workforce has the training to keep up with 
labor market demands is difficult, given the wide 
range of regional and state needs and the pace of 
economic change in many regions. At a minimum, 
supporting a well-trained, well-educated workforce 
requires an understanding of the specific 
knowledge and skills necessary for any given 
job, the development of learning opportunities 
that align with those needs, and mechanisms 
to connect workers with that training. Often, 
employers know best which skills are needed 
in the workplace, yet high school and college 
instructors are the ones developing and delivering 
the training curricula. Unfortunately, these two 
groups can find it challenging to work together. 

The California Community Colleges’ (CCC) 
Economic and Workforce Development Program 
(EWD) was first created in 1982 and became 
a state statute in 1991 . EWD aims to support the 
development of a workforce that will promote 
California’s economic development by connecting 
employers and educators. The 2012 legislation 
that reauthorized EWD (SB 1402, Chapter 361, 
Statutes of 201 2) established a revised program 
that will sunset in 201 8. 3 The law makes clear 
that EWD should have both statewide and 
regional strategies and lists services suitable 
for EWD to provide, including convening skill 
panels, developing instructional packages, faculty 
professional development, one-on-one counseling 
for businesses, job training, subsidized student 
internships in priority industry sectors, and facility 
renovations. 


SB 1402 also outlines performance accountability 
measures the California Community Colleges 
Chancellor’s Office (CCCCO) must gather, and it 
requires EWD’s effectiveness to be evaluated. This 
report summarizes the findings of that evaluation. 

Our report is organized as follows: first, we 
describe EWD’s, structure and its role in the 
CCC system. Next, we provide an overview of 
the evaluation’s purpose and objectives, along 
with its design, methodologies, and related 
limitations. Then, we present findings, organizing 
them into three broad areas: how EWD grantees 
execute their roles; program outcomes; and an 
assessment of EWD’s impact and effectiveness. 
We then present recommendations for program 
improvements; these reflect both our analyses 
and suggestions from EWD grantees and 
stakeholders. We conclude with final thoughts 
on EWD— its impact and implications for 
California’s workforce and community colleges. 

EWD Program Structure 

SB 1402 provides a broad mission, objectives, 
and strategies for EWD, but the program’s design 
and execution ultimately is left to the CCCCO. 

In an effort to meet the requirements set 
out by the legislature, the CCCCO crafted 
EWD’s specific mission to be to “invest in 
California’s economic growth and global 
competitiveness through industry-specific 
education, training, and services that contribute 
to a highly skilled and productive workforce.” 

EWD’s new structure centers around a regional- 
and sector-focused service delivery structure. 

The CCC system uses regional consortia to 
bring together CTE, economic development, 
workforce development, and contract education 
programs. Each regional consortium is led by 
a chair who participates in activities that promote 
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the colleges’ role in advancing economic growth. 
This regional focus means that EWD services 
are tailored specifically to each of 15 unique 
geographic regions of California. These areas are 
shown in Figure 1 . In contrast a statewide focus 
treats the rural Mother Lode the same as urban 
Silicon Valley. Moreover, rather than focusing on 
all sectors of the economy, EWD selected 10 
high growth sectors, also shown in Figure I. 4 

EWD grantees identified these sectors of the 
economy as high priority due to their rapid 
growth, high demand for workers, and well- 
paying jobs. Each region selected five of the 10 
sectors— three priority and two emergent— to 
focus on. For example, in the East Bay Region, 
advanced manufacturing, health, and life sciences/ 
biotechnology were chosen as priority sectors 
and advanced transportation and renewables and 
information and communications technology (ICT)/ 


digital media as emergent sectors. EWD developed 
a new approach to regional and sector staffing as 
follows (roles and services offered by the grantees 
are discussed further in the Findings section): 

Sector navigators (SNs) serve as experts in the 
sectors and on community college program 
offerings related to those sectors. In their role, 
the 10 SNs coordinate and support deputy sector 
navigators (DSNs), who help the CCC system 
connect with major employers and employer 
groups and guide efforts to improve workforce 
development in their respective sectors. 

Deputy sector navigators address their sectors’ 
unique regional needs. Each of seven macro 
regions generally has one DSN assigned to every 
sector designated as priority or emergent by the 
micro regions. At times, however, the CCCCO 
provided regions with additional DSNs as needs 


Figure 1 . EWD Program Regions and Priority Sectors 



Source: California Community Colleges' Economic and Workforce Development Website (http://cccewd.net/) 
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arose. For example, the CCCCO reported receiving 
extra funds for the global trade and logistics 
sector, and as a result was able to provide an 
extra DSN to regions that had not originally 
selected global trade and logistics as a priority 
or emergent sector. At the time of this report, 
there were 66 DSNs working with the community 
colleges, employers, and other key stakeholders 
in workforce training to promote EWD objectives 
of improving economic growth and workforce 
development. Working at the macro-regional level, 
DSNs aim to ensure alignment of employer needs, 
community college offerings, and career pathways. 

Technical assistance providers (TAPs) provide 
support to help the SNs and DSNs fulfill their 
roles. The EWD-funded TAPs are the Centers 
of Excellence for Labor Market Research 
(CoEs), LaunchBoard, contract education, and 
communications/meetings. The seven CoEs 
serve their macro region by providing SNs and 
DSNs with labor market information to support 
data-based decision making. LaunchBoard is 
a data system that tracks the progression and 
outcomes of CCC career technical education 
(CTE) students, with the goal of supporting 
conversations about student success and continual 
program improvement. Contract education is 
supported privately by employers rather than 
through state funds, with community colleges 
providing training to employers’ workers; this 
generally allows for training to be developed 
quickly and for the specific needs of the employer. 
Communications/meetings provide support for 
organizing EWD grantee meetings and events. 

Industry-driven regional collaboratives (IDRCs) are 

short-term (two years or less) projects that address 
specific regional workforce needs. IDRC directors 
may have other roles in the CCC system, such as 
being EWD SNs. Currently, there are 12 IDRCs. 

EWD plays a key role in the CCC’s Doing What 
Matters for Jobs and the Economy (DWM) initiative. 
This initiative incorporates several CTE efforts 


of the CCC, such as the state-funded Career 
Technical Education Pathways Program (SB 
70, Chapter 433, Statutes of 2012), the federal 
Carl D. Perkins Career and Technical Education 
Act of 2006, and state-funded apprenticeship 
programs. DWM’s stated goal is to “supply in- 
demand skills for employers, create relevant 
career pathways and stackable credentials, 
promote student success and get Californians into 
open jobs.” The CCCCO, in implementing EWD, 
integrated it into the larger DWM initiative. 5 The 
impact of using the DWM framework for EWD 
is discussed in the section on EWD strengths. 


DWM Framework 

1 . Give priority to jobs and the economy. 

• Consider labor market needs when 
making budget, course, and program 
decisions. 

• Decide on program capacity as a 
region. 

2. Make room for jobs and the economy. 

• Retool programs that are not working 
or meeting a labor market need so that 
students can study what matters. 

3. Promote student success. 

• Adopt common metrics and skills 
panels in grants administered by 
the CCCCO. 

• Strengthen regions with new skill sets. 

4. Innovate for jobs and the economy. 

• Solve complex workplace training 
needs so that the community college 
system can better deliver to employers 
and sectors. 
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Overview of Evaluation Methodology 
and Data 


The evaluation took eight months. During that 
time, we created interview and survey instruments, 
developed a sampling plan, collected new data, 
received access to existing data files, transcribed 
interviews, and analyzed all data in an effort to 
answer research questions posed by the CCCCO 
(see Appendix C for the list of research questions). 
To inform the program evaluation, we analyzed 
qualitative data using content analyses to uncover 
dominant themes. Quantitative data were examined 
using descriptive statistics. We were unable to use 
inferential statistics due to data limitations (see 
text box for an overview of the data; see Appendix 
B for details on the evaluation methods and data). 

While we were able to gather rich information from 
EWD grantees, we faced significant barriers in 
other aspects of the evaluation. The quick timeline 
of this evaluation limited our capacity to gather 
information from stakeholders. The design of EWD 
and our lack of access to relevant data forced us 
to make many concessions in reporting outcomes. 

Estimating the impact of a program is difficult 
under any circumstances. In this instance, it was 
not possible to quantify causal impacts. First, 
participant-level data on EWD service recipients did 
not exist. Second, we could not randomly select 
and assign which students, faculty, staff, deans, 
grantees, and other participants and stakeholders 


would receive or be influenced by program 
services. Third, we only had access to aggregate 
data and not student-level data; there is no way to 
identify “EWD students” (or a comparison group of 
non-EWD students) in the CCC’s data system, as 
EWD funds are not directed to specific programs 
or students but rather are integrated with other 
funding sources to support CTE programs more 
generally. Given these limitations, our findings 
should not be interpreted as impacts or effects 
of EWD. However, we do report perceptions of 
impact throughout the qualitative findings. 


Data sources 

DSN interviews (n=25) 

SN interviews (n=10) 

TAP: CoE director interviews (n=7) 

TAP: IDRC interviews (n=3) 

TAP: LaunchBoard interview (n=1) 

DSN survey (n=41) 

Regional consortia chair survey (n=8) 

CTE deans survey (n=21) 

Employer survey (n=40) 

CCCCO Management Information System data 
EWD grantees’ quarterly reports (n=97) 
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Findings 


This evaluation aimed to understand not only 
impact, but the processes by which EWD executed 
its mission. In this section, we discuss our 
evaluation findings. First, we discuss how EWD 
grantees executed their roles to meet program 
objectives. We focus on grantee roles, the services 
and activities they provided, how they coordinated 
with each other and non-EWD stakeholders, 
how they used their funding, and how they used 
data. Next, we present perceived outcomes for 
students, employers, and education and workforce 
organizations. We conclude with an assessment 
of EWD strengths and areas of challenge. 

How Grantees Aimed to Meet 
Program Objectives: Roles, Services, 
Coordination, Funding, and Data Use 

The following findings show how EWD grantees 
aimed to meet program objectives. We analyzed 
the grantees’ roles, the services and activities 
they provided, how they coordinated with each 
other and with other EWD stakeholders, how 
they integrated funding from non-EWD sources, 
and how they used student data and labor 
market information. Our main findings were: 

• Roles. DSNs were seen as most important 
in providing services, with their primary 
focus being (1) connecting employers and 
community colleges and (2) assisting in 
aligning curriculum. 

• Services. Grantees, including DSNs, provided 
2,577 services to employers and community 
colleges in 2013-14 and served 13,723 
employers, 41 ,252 students, and 38,486 
employees. 

• Collaboration. Coordination occurred 
intentionally and frequently among EWD 
grantees and between EWD grantees and 
other EWD stakeholders. 

• Funding. Grantees report braiding various 
funding sources in support of economic and 
workforce development. 


• Data use. Grantees reported trying to use data 
to support decision making, but struggling to 
do so due to a lack of current, relevant, and 
accurate data. 

Roles 

EWD focused on 1 0 priority sectors and aimed 
to support economic development by delivering 
services regionally. As such, each priority 
sector had a statewide lead, the SN. Each of 
the seven regions selected sectors to focus on. 

The sector leads in a region were the DSNs. 

CoEs provided labor market information and 
supported labor market research efforts with 
and for EWD. Four TAPs provided support to 
different aspects of the program, such as for 
contract education or LaunchBoard. Finally, 

IDRCs focused on providing a specific program 
to support workforce development and training. 

In this section, we discuss the roles of each of 
the EWD grantees. Specifically, we describe 
the nature and quality of services provided to 
colleges, businesses, students, and employees. 

We begin with DSNs, who most directly 
deliver services to EWD stakeholders. 

Deputy sector navigators focused mainly on 
connecting employers and community colleges 

DSNs described their main purpose as increasing 
economic development and helping the economy 
of their regions through workforce training. They 
accomplished this by identifying workforce 
needs, recruiting students and workers for 
training, and working with community colleges 
to develop employer-valued workforce training. 
DSNs reported spending about 50 percent 
of their time on EWD (SB 1 402) activities, 30 
percent on CTE pathways (SB 1 070) activities, 

10 percent on host college tasks, and 7 percent 
on other activities, such as responding to emails 
and attending trainings. They reported that they 
spend, on average, 48 percent of their time 
working on industry activities, 42 percent on 
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college activities, and 1 0 percent on outreach 
to relevant organizations and high schools. 

Their work focused mainly on connecting 
employers and colleges, and on aligning 
curricula with workforce needs. They also 
spent significant time helping to advance 
career pathways. The DSNs reported spending 
time as liaisons among community colleges 
and serving as legislative advocates. These 
roles are described in greater detail below. 

Connecting employers and community colleges 
DSNs said their most important role was as active 
liaisons between the community colleges and 
the employers in the regions and sectors they 
covered. They reported connecting employers 
and colleges in three main ways (see Table 1): 

• Building relationships between 
businesses and colleges by acting as 
intermediaries and connectors. 

• Helping employers navigate 
community colleges in the region. 

• Placing community college students into jobs. 

They described meeting directly with employers 
to understand their labor needs (with 28 percent 


of their time spent building relationships with 
businesses and 18 percent spent providing 
technical assistance to businesses) and meeting 
directly with community colleges to communicate 
these needs. DSNs also reported facilitating 
conversations between employers and community 
colleges around workforce development. CTE 
deans saw the role of the DSN as the most useful 
support provided by EWD for its CTE programs. 

For regions with multiple community colleges, 
DSNs also helped employers understand programs 
available in each region, so that employers 
wouldn’t need to seek the information college 
by college instead. For example, DSNs reported 
developing curriculum committee teams to 
ensure consistency across the region. Unlike 
the regional consortia, these teams aimed to 
ensure that similar programs across the region 
provided comparable training so employers 
could understand a program graduate’s skill set. 
Another DSN reported developing marketing 
materials for the region’s community colleges. 
Serving as liaisons between employers and 
community colleges, DSNs also reported 
placing students into jobs. Aware of the skill 


Table 1 . Percent of DSNs Believing Their Services Contributed to a Highly Skilled and Productive Workforce 


To Colleges 


To Businesses 


Building relationships with business/ 
industry 

92% 

Connecting with colleges and education 

95% 

Providing professional development 
for faculty 

81% 

Providing professional development 

for workers 

86% 

Providing technical assistance to 
business/industry 

75% 

Aligning business sector with education 

78% 

Developing new and/or revising 
programs of study 

58% 

Providing assessments (such as 
business needs, technology needs, 
manufacturing process capability) 

60% 

Providing research and studies 

55% 

Providing research and studies 

50% 

Developing new and/or revising courses 

47% 

Creating small businesses and/or 
exporting modules 

32% 


Source: EWD Program Evaluation Survey of DSNs. 
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sets being developed by the programs in their 
sectors and regions, and aware of the demands 
of employers, DSNs found themselves well- 
positioned to help students with career placement. 
In their role as DSNs, they felt they represented 
both industry and the community colleges. 

Aligning curricula 

Using the expertise built from their relationships 
with employers, DSNs noted that one of 
their top services was aligning community 
college and employer needs. In this role, 

DSNs helped match curricula for students 
with employer-valued competencies by: 

• Working collaboratively with faculty to 
update and revise curricula and programs. 

• Providing needed resources, such as 
equipment, professional development, and 
funding, directly to faculty and departments. 

Forty-seven percent of DSNs reported that guiding 
the development of new and/or revised courses 
was a very important part of the services they 
provided to contribute to building a highly skilled 
and productive workforce (see Table 1). Of the CTE 
deans surveyed, 56 percent said EWD assisted 
them in revising and improving program curricula, 
44 percent said EWD helped them create new 
program curricula, and 12 percent said EWD 
aided their elimination of outdated programs. As 
to the services they would find helpful in aligning 
curriculum, 95 percent of CTE deans surveyed 
listed obtaining assistance in revising and 
improving program curriculum, and 90 percent 
cited getting support to create new program 
curriculum. However, a small number of CTE deans 
said these tasks were the purview of the faculty. 

DSNs also provided the community colleges 
in their regions with resources, including 
equipment, software, professional development 
programs, and funding. Seventy-eight percent 
of CTE deans surveyed said EWD grantees 
had positively impacted CTE programs on 
their campuses by supporting professional 
development programs for faculty and staff. 


Offering trainings 

DSNs frequently mentioned that a key service they 
provided was training for instructors, students, 
incumbent workers, and employers. Their overall 
goal was to increase economic growth through 
workforce development, and the training content 
varied depending on the groups being targeted. 
Examples include training targeted for: 

• Instructors focused on how to teach 
a new industry certification. 

• Students seeking to develop soft skills. 

• Incumbent workers asked by employers 
to acquire job upskilling. 

• Employers hoping to grow their footprint 
in the region or internationally. 

DSNs’ work as trainers resonated most with CTE 
deans, who consistently noted that EWD supported 
professional development for faculty and staff. 
Deans rated improving students’ preparation for 
jobs in their chosen fields of study as the most 
important skill the deans would like to improve (95 
percent rated student job preparation training as 
“very important”; 5 percent as “important”). The 
deans also highly rated increasing support for 
students to receive more up-to-date job training 
and greater proficiency with current technology. 

Advancing career pathways 
DSNs generally stated that they helped advance 
career pathways from middle school to community 
college in two ways: (1) by working with faculty to 
develop curricula and (2) by creating marketing 
materials to build awareness of career pathways. 
With regard to developing curricula, they reported 
working with faculty at all grade levels to create 
curricula that promoted entry to career fields 
in each sector the DSNs represented. The 
DSNs brought together faculty from different 
educational levels to collaborate on creating 
connected pathways and formal articulation 
agreements. In building awareness, DSNs 
described information sheets that they created 
about jobs in their sectors. These marketing 
materials were used to help students beginning 
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in middle school to understand their career and 
training options and to help faculty, counselors, 
and advisors guide students more successfully. 

Acting as liaisons among community colleges 
In addition to connecting employers with 
community colleges, DSNs sought to facilitate 
collaboration among California’s community 
colleges. DSNs said they aimed to use their 
knowledge of program offerings across their 
regions to share information among community 
colleges. This included information sharing about 
curriculum and course offerings for faculty. 

Being legislative advocates 
Less often, DSNs mentioned their role as 
policy advocates for community colleges and 
for their regions’ sectors. Attending legislative 
hearings and providing feedback on bills were 
two activities grantees consistently reported. 

Sector navigators served as experts 
on their sectors’ needs 

Sector navigators saw their role as identifying 
the needs of their sectors concerning 
sector growth and the need for skilled 
employees. The SNs sought to do this by: 

• Being expert resources for employers, 
community colleges, and DSNs about 
sector needs by convening interest 
groups and working with major 
employers to build networks. 

• Guiding and supporting DSNs through 
the provision of resources and 
information inaccessible to DSNs. 

• Increasing employer awareness of the 
role of community colleges in workforce 
training through holding one-on- 
meetings, developing marketing materials, 
and attending industry events. 

Industry-Driven Regional Collaboratives 
Focused on Specific Projects 

IDRC directors described their role as ensuring 
success in their specific workforce training 


projects. However, these projects were not closely 
connected to the core functions of EWD’s SNs and 
DSNs. IDRC directors interviewed also held other 
roles, such as SNs, which they seemed to view as 
their primary responsibility, as they only discussed 
their IDRC role when specifically prompted. 

Centers of Excellence provided labor 
market information to grantees 

CoE directors identified their role as providing 
technical assistance and labor market research 
to the California Community Colleges and 
EWD grantees in their regions. They sought 
to understand the needs of their regions 
by taking requests for assistance from the 
groups and individuals they supported 
and by surveying their colleges. 

As labor market experts, they conducted employer 
surveys and focus groups and led marketing 
research efforts for the colleges. The goal of 
their analyses was to understand employment 
opportunities for community college students 
and to examine employer workforce needs. 

As technical assistance providers, the CoE 
directors assisted faculty, CTE deans, regional 
consortia, EWD grantees, and other Doing What 
Matters for Jobs and the Economy grantees in 
understanding labor market information to help 
them close skills and training gaps. To this end, 
the CoE directors held workshops, presentations, 
and individual sessions. Their audiences were 
representative of the variety of stakeholder 
groups across these directors’ regions. 

LaunchBoard aimed to provide data in support 
of accountability and program improvement 

The technical assistance providers developing 
LaunchBoard said their role was building a data 
system that will allow community colleges 
to have data-based conversations about the 
effectiveness of CTE programs, as well as 
make decisions based on data to improve 
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student outcomes both in college and in the 
labor market. LaunchBoard worked across 
CTE, including EWD, to understand data needs 
and provide training on using the system. 

Services and Activities 

In their roles described above, EWD grantees 
reported providing 14 different services to 
colleges and employers (see Tables 2 and 
3 for a breakdown of all services provided by 
EWD grantees). In total, EWD grantees: 

• Provided 2,577 services to 
colleges and businesses. 

• Served 13,723 businesses, 41,252 
students, and 38,486 employees. 

These services were comprised of: 

• 32,935 hours of workshops and training. 

• 1 1 6,906 hours of contract education. 


• 64,866 hours of performance 
improvement training. 

• 115,146 hours of credit/non-credit instruction. 

We found that the grantees were active in terms 
of the quantity and breadth of services they 
provided to colleges, students, employers, and 
employees. While there was a slight variation 
in the quantity of services provided by sectors 
and regions, we did not find any consistent 
differences that would suggest one sector 
provided significantly more or different services 
than another. However, variation was present in 
the types of services each grantee provided. 


Table 2. Total Grantee Services to Colleges - Number of Times Grantees Provided Each Service 


Grantee 

Develop & Align 

Curriculum 

Curriculum 

Alignment 

Certificate & 

Program of Study 
Development 

Career Curriculum 

Articulation 

Faculty Professional 
Development 

Connect to 

Businesses and/or 

Industries 

Research & Studies 

DSN 

186 

103 

142 

117 

214 

245 

134 

SN 

37 

13 

17 

20 

38 

43 

23 

CoE 

0 

0 

0 

0 

0 

0 

22 

IDRC 

22 

19 

9 

11 

9 

28 

1 

Contract Ed 

0 

0 

0 

0 

1 

1 

0 

Weighted Average 

134 

75 

101 

84 

152 

176 

96 

Quarterly Average 

22 

12 

17 

14 

36 

29 

16 

Total 

245 

135 

168 

148 

262 

317 

180 


Source: 2013-2014 Grantee Quarterly Reports. 
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Table 3. Total Grantee Services to Businesses - Number of Times Grantees Provided Each Service 


Grantee 

Develop Curriculum 

for Business 

Alignment of Sector 

with Education 

Needs Assessment 

Small Business 

Creation and/or 

Exporting Modules 

Professional 

Development for 

Workers 

Connect with 

Colleges and 

Education 

Research 

DSN 

97 

149 

159 

49 

130 

223 

96 

SN 

15 

27 

22 

9 

15 

32 

11 

CoE 

0 

0 

0 

0 

0 

0 

4 

IDRC 

9 

8 

13 

0 

16 

23 

0 

Contract Ed 

0 

0 

0 

0 

1 

1 

0 

Weighted Average 

69 

106 

113 

35 

93 

160 

67 

Quarterly Average 

12 

18 

19 

6 

15 

27 

11 

Total 

121 

184 

194 

58 

162 

279 

111 


Source: 2013-2014 Grantee Quarterly Reports. 


Different grantees offered a variety of services 

In total, EWD grantees provided 2,577 services 
to all types of recipients. The most common 
services connected employers and colleges 
(402 services per year) 6 , conducted professional 
development (306 per year), and aligned curriculum 
and sectors (208 per year). SNs and DSNs 
provided the most services per grantee, 5.7 and 
5.2 each quarter, respectively (see Table A-1). 

The most common services DSNs and SNs 
provided to colleges were connecting them to 
businesses, conducting faculty professional 
development, and developing and aligning 
curriculum (see Tables 2 and 3). The most 
common services given to employers were 
helping businesses connect to colleges, 
assessing business needs, and connecting the 
business and education sectors (see Tables 
3). DSNs provided the greatest number of 
services of all grantees, even when taking 
into account the large number of DSNs. 


In fiscal year 2013-14, IDRC grantees provided 
99 services, and the most common ones were 
similar to those offered by DSNs and SNs. But 
instead of providing professional development 
to colleges, IDRC grantees aligned curricula with 
a third party, such as state licensing boards or 
industry credentialing standards. Still, the bulk of 
the grantees’ work in 2013-14 was conducting 
credit/non-credit classes and teaching 1 ,949 hours. 
Those teaching hours are still less than those 
provided during the same year by DSNs and SNs. 

The two grantees that offered the most 
focused services were CoE (22) and contract 
education (4) grantees. CoE grantees focused 
on offering research services, while the 
contract education grantees aimed to provide 
professional development and to connect 
colleges and employers. However, the contract 
education grantees taught the highest average 
number of hours (5,597) and served half the 
employees who received EWD services. 
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The number of services offered varied 
by sector, less so by region 

While most services each sector provided tended 
to fall into the three main categories— connecting 
employers and colleges, providing professional 
development, and aligning curriculum and 
sectors— some variation among sectors was 
present (see see Appendix A, Table A-5). 

The advanced manufacturing sector provided 
the highest average number of services for 
curriculum alignment and for connecting colleges 
with employers, as well as for developing 
curriculum for businesses and conducting needs 
assessments. This sector also had the highest 
average and highest total number of workshop 
completions and of hours for contract education, 
credit/non-credit courses, and performance 
training. With the exception of performance 
training, the advanced manufacturing sector 
was responsible for over 90 percent of the 
combined hours worked by EWD grantees. The 
advanced transportation and renewables sector 
had the highest average number of services for 
providing research to colleges and professional 
development to employers. Lastly, the health 
sector had the highest average number of services 
for professional development that were used by 
colleges and to connect employers with colleges. 

Some sector grantees were more productive 
than others. Both the health and the energy and 
utilities sectors provided the highest number of 
services per grantee, an average of 35. The global 
trade and logistics sector had one of the highest 
numbers of grantees among the regions (11), and 
the smallest number of services per grantee (27). 

Given that the DSNs are located throughout 
California in 15 micro regions and seven macro 
regions, we examined regional differences (see 
Appendix A, Table A-6) in terms of the services 
they provided. We found that the number of 
services provided by region did not vary as much 
as it differed by sector. The South Central Coast 


provided the highest number of services per 
DSN (37) and also the highest average for seven 
different kinds of services. The two regions that 
provided the smallest number of services per DSN 
were North Coast/lnland and Greater Sacramento 
(25), and the Mother Lode and Central Valley (20). 

Coordinating Efforts Around Economic and 
Workforce Development 

EWD grantees spent significant time and effort 
coordinating among themselves and with 
stakeholders, including workforce agencies, 
employer groups, and educational institutions. 
Regular, formal meetings were common among 
DSNs by region and sector, between sector SNs 
and DSNs, and among various stakeholders by 
sector and region. In general, EWD grantees 
found that coordinating efforts was easier 
with educators than with employers. Grantees 
reported that employer staffing turned over 
quickly, making it difficult to build relationships. 
Additionally, they reported employers were 
more guarded and less open to coordination 
efforts than were colleges. (Further details on 
coordination can be found in Appendix E.) 

Among EWD grantees, we found that: 

• SNs and DSNs worked closely with 
each other to advance their sector. 

• DSNs in the same sector or region 
had strong coordination. 

• Coordination between SNs and DSNs 
with CoEs varied substantially. 

• IDRCs built coordination around 
a specific project. 

Coordination with entities outside EWD was also 
viewed as a priority and was particularly strong 
for other partners in the DWM initiative, such as 
the regional consortia. We also learned that: 

• Regional consortia was a key coordinating 
structure for regional CTE activities. 
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Table 4. DSN and SN Total Number of Services by Sector 
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Advanced Manufacturing 
& Advanced Technology 


20 


26 


21 


23 


41 


14 


22 


18 


28 


23 


35 


290 


32.22 


12% 


Advanced Transportation 
& Renewable Energy 


16 


13 


18 


19 


14 


11 


14 


11 


17 


18 


171 


34.20 


7% 


Agriculture, Water 
& Environmental 
Technology 


20 


13 


19 


15 


24 


22 


12 


17 


12 


16 


192 


32.00 


8 % 


Energy (Efficiency) 
& Utilities 


13 


14 


13 


17 


11 


14 


11 


17 


140 


35.00 


6 % 


Global Trade & Logistics 


24 


15 


13 


33 


35 


15 


19 


25 


26 


18 


19 


29 


24 


301 


27.36 


13% 


Health 


32 


17 


18 


19 


50 


49 


26 


16 


33 


27 


37 


48 


16 


389 


35.36 


16% 


Info & Communication 
Technologies/ 

Digital Media 


38 


17 


31 


29 


34 


37 


23 


21 


20 


10 


37 


13 


320 


29.09 


13% 


Life Sciences / 
Biotechnology 


12 


10 


12 


83 


27.67 


3% 


Retail/Hospitality/ 

Tourism 


12 


13 


18 


11 


12 


12 


16 


11 


134 


33.50 


6 % 


Small Business 


36 


14 


25 


23 


35 


38 


23 


16 


21 


33 


28 


32 


26 


359 


29.92 


15% 


Total 


223 


116 


159 


137 


252 


288 


157 


113 


178 


184 


58 


146 


257 


111 


2379 


31.30 


1 00 % 


Source: 2013-2014 Grantee Quarterly Reports. 


• EWD grantees built on existing relationships 
and developed new employer relationships 
to coordinate workforce training. 

• Linking SB 1 070 and AB 1 402 funds 
drove EWD coordination with K-12. 

• CCCCO both supported and 
unintentionally impeded coordination 
through its management of grants. 

• Successful coordination at community 
colleges hinged on several factors 
beyond the control of grantees. 

• EWD grantees and workforce organizations 
regularly coordinated, but diverging 
performance metrics were a barrier. 

Funding 

EWD grantees braided together other funding 
sources to support economic and workforce 
development. Receiving SB 1070 funds or 
working with SB 1070 teams was commonly 
noted by DSNs. These funds supported DSNs’ 
efforts to promote the sectors in their regions’ 

K-1 2 career pathways. DSNs expressed that they 
felt successful in their work to support career 
pathways. They reported using Proposition 
39 and CTE Enhancement Funds to pay for 
community colleges to collaborate together, 
create professional development programs, and 
fund equipment upgrades. One DSN noted using 
federal funds to pay for one-on-one training for 
business leaders (e.g. prepping for a trade show). 
CoE directors reported receiving funding from the 
federal Workforce Innovation and Opportunity Act 
(WIOA) and from California CTE Enhancement 
Funds. The WIOA funding supports the regional 
partnerships between state Workforce Investment 
Boards and community colleges. CoE saw CTE 
Enhancement Funds as an opportunity to acquire 
additional funding that was not for a specific task. 


Data Use 

SNs and DSNs reported trying to use data to 
help inform their decisions about services to offer 
and that their most common source for labor 
market reports were the Centers of Excellence. 
They most frequently accessed data to: 

• Understand market trends (88 percent). 

• Understand basic demographics (85 percent). 

• Decide whether to add a new 
program (78 percent). 

• Revise a program (70 percent). 

According to DSNs, however, not all the data they 
accessed was useful in their decision-making. 

Most DSNs (97 percent) found CoE reports useful, 
while 32 percent found LaunchBoard useful 
(see Table 5). In interviews, grantees indicated 
that LaunchBoard’s goal of providing student- 
level data to help understand student progress 
would be useful, but most grantees expressed 
frustration that LaunchBoard had struggled 
with data accuracy and lack of data usability. 

Mixed perceptions arose about the usefulness 
of Centers of Excellence labor market research 

SNs and DSNs reported using labor market 
information provided by CoE, and 97 percent of 
DSNs found CoE reports to be useful, but grantees 
interviewed also provided critical feedback 
on how the data’s utility could be improved. 

It appears that some individuals at CoE were 
more adept than others at working with SNs 
and DSNs to understand their needs and deliver 
analyses to address those needs. SNs and DSNs 
said that some CoEs were unable to provide 
useful information or were not able to market 
their services well enough for SNs and DSNs to 
understand the usefulness of this assistance. 


Table 5. Data Sources Rated as Useful or Somewhat Useful by DSNs 


LaunchBoard 

MIS 

DataMart 

Burning Glass 

TAP research and reports 

32% 

50% 

58% 

62% 

97% 


Source: EWD Program Evaluation Survey of DSNs (n=38). 
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SNs and DSNs who did use CoE services reported 
being very satisfied with the data they received. 
They also said that when existing data were not 
sufficient, CoE worked with them to create and 
administer original surveys to provide them with 
the information they needed. They described how 
CoE information helped them identify labor market 
needs (including shortages and necessary skill 
sets) and to assist faculty and deans in reviewing 
curricula in relation to labor market needs. CoE 
reported that its reports led to improvements 
in hundreds of community college classes. 

Those who identified challenges with CoE data 
highlighted the difficulties of obtaining data specific 
and recent enough to be actionable. Other data 
challenges included a lack of labor market data 
for EWD sectors not aligned with federal Standard 
Occupational Classification codes and problems 
capturing data for self-employed workers. 

LaunchBoard’s flawed introduction 
left grantees disengaged 

EWD grantees reported that LaunchBoard was 
introduced prematurely, leaving grantees cynical 
about its usefulness. The premature launch 
revealed problems with the data system, including: 

• The system was difficult to navigate. 

• Some data were incomplete, 
dated, or irrelevant to EWD. 

• The way the data were aggregated 
limited the analyses they could conduct 
and questions they could answer. 

• The coding of courses was not 
aligned with grantees’ understanding 
of CTE and EWD courses. 7 

• DSNs were responsible for entering 
data they could not access. 

• Data definitions were unclear. 

• DSNs did not have enough time to input data. 

• DSNs had inadequate LaunchBoard training. 

• The data’s usefulness to SNs 
and DSNs was unclear. 


For these reasons, DSNs almost universally said 
they did not use LaunchBoard. They acknowledged 
that LaunchBoard was a work in progress, but 
were hesitant to spend their limited time to 
log back in until it was fully developed. Others 
were discouraged because they had attended 
multiple trainings and heard LaunchBoard would 
be functioning soon, but were still waiting. SNs 
were particularly negative about LaunchBoard 
as a resource. They described it as not worth 
the money and time, particularly the time 
DSNs would have to spend inputting data. 

Grantees looked for other data, but had 
unmet needs 

Most grantees reported depending solely on the 
data provided by EWD and the colleges. Some 
DSNs, however, went directly to government 
sources of data, such as the U.S. Department 
of Labor, the U.S. Department of Commerce, 
the California Employment Development 
Department, and government sector-specific 
sources (e.g. the Visit California website for 
tourism data). DSNs also said they sought and 
received data from their employer partners. 

In terms of their unmet data needs, EWD grantees 
stated that they needed better information to help 
community colleges determine what courses to 
offer or revise based on labor market changes. 

They indicated that many colleges seemed 
more concerned about generating revenue 
rather than improving the workforce. DSNs 
said data on labor trends and about workforce 
training needs could help them encourage 
the colleges to make curriculum changes. 

Secondly, DSNs reported needing better 
information to understand their impact on 
economic and workforce development. About 
70 percent said they need more data— data to 
specifically help them answer questions about 
student labor market and industry outcomes (e.g. 
earnings and hiring) related to the DSNs’ own work. 
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Perceived Outcomes for Students, 
Employers, and Workforce Entities 

In the 2013-2014 fiscal year, EWD grantees 
reported that 41 ,252 students received services. 
While we cannot conclude that these services 
helped students attain better educational 
outcomes, we found students performed better 
during this same time period. Students passed 
courses at higher rates, and more certificates and 
degrees were awarded. 8 Similar to these students, 
grantees reported that employers who received 
EWD services experienced positive outcomes. 
Employers surveyed who knew of the EWD 
program stated that they found EWD services to 
be useful. Yet many of the employers surveyed 
were unaware of the services EWD provided. 

Perceived Student Outcomes 

In their quarterly reports, EWD grantees 
reported that they placed: 

• 1,135 students or trainees in jobs. 

• 2,650 students into work-based 
learning programs, such as 
apprenticeships or internships. 

In surveys and interviews, SNs and DSNs said 
their main outcome was improving students’ job 
prospects. Grantees also said students were more 
aware of their career options. They claimed this 
was mostly due to their efforts at the high school 
level to market their sectors. However, half of CTE 
deans surveyed indicated that students were more 
interested in programs in specific sectors that 
EWD targets. This may be due to the marketing 
efforts of EWD, or to the growing popularity of EWD 
sectors that are growing and involve higher-earning 
and higher-demand fields. While we did not have 
student evidence to validate grantee perspectives, 
other data indicate that changes were occurring 
at the community colleges. For example: 

• Full-time-equivalent student (FTES) 
enrollment was declining in CTE courses 
in the community colleges, and these 


declines also were reflected in courses 
identified as being in EWD sectors (we 
will refer to these as “EWD courses”). 9 

• Students achieved higher passing 
rates in EWD and CTE courses. 

• Students earned more associate degrees and 
certificates in EWD-related and CTE fields. 

FTES’ Enrollment in CTE Courses Dropped 

Over the past few years, the number of FTES 
in EWD courses has dropped (see Appendix A, 
Tables A-7 and A-8), with a 12.8 percent decline 
between 2010 and 2015. This decline mirrors CTE 
enrollment, as CTE experienced a 12.7 percent 
enrollment decline during the same time period. 
The ICT/digital media sector experienced 
the largest loss, 13,191 FTES (a 19 percent 
decrease), between 2010 and 2015. The advanced 
transportation and renewable energy sector had 
the smallest percentage loss at 6.1 percent, or 
a decrease of 875 FTES. Community college 
FTES enrollment dropped overall during 
those years with a 7.3 percent decline. 10 
Passing rates increased for EWD courses 
We did not have data on the competencies 
attained by EWD participants, but we did have 
data on students’ course passage rates. We used 
this as a proxy for workforce competency, since 
a passing grade indicates a proficient level of skill 
in the course completed. To create this category 
of proficiency, we included students who had 
achieved a grade of C or higher or passed a pass/ 
fail course. From 2010 to 2015, the passing 
rates for EWD courses increased modestly by 
2 percent. But upon further examination, we 
found that students in CTE fields that were not 
related to EWD sectors received better grades 
than students taking EWD courses. The course 
passing rate for students in CTE fields not 
related to EWD sectors in 2013-2014 was 81 
percent (and had increased by 1 .4 percent since 
2010). The same year, the average passing rate 
for those in EWD courses was 74 percent. 
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EWD program awards increased 

Program completion is another measure of 
student success. While we did not have data 
on students’ rate of completing CTE programs, 
data on the number of degrees and certificates 
awarded indicated that since 2010, the number 
of chancellor’s office-approved awards in 
EWD-related fields increased each year. 11 In 
fact, 22 percent (or 9,446) more awards were 
granted in 2014-2015 than in 2009-2010. We 
focused on the 2013-2014 school year, since 
it was the first year the DWM framework was 
implemented. During that year, 51,032 awards 
were granted. Of those awards, the majority 
were AA degrees (52 percent), with certificates 
accounting for the remaining awards. Of those 
awards, small business accounted for the 
most AA degrees awarded (39 percent). 

Health awarded the second most degrees (32 
percent). Interestingly, health was the only sector 
to experience a decline in the number of AA 
degrees awarded. All other sectors— except for 
life sciences/biotechnology, in which the number 
of degrees awarded stayed relatively flat — 
experienced an increase in AA degrees awarded. 

Of the certificates earned in 2013-2014, 46 percent 
required 30 to 60 semester units (one to two 


years, if the student was full time), and 42 percent 
required 1 8 to 30 semester units (one semester 
to one year, if the student was full time). Only 
4 percent of certificates earned required more 
than 60 semester units. We saw similar trends 
for awards in non-EWD CTE fields. Over the past 
five years, students have been earning more 
degrees, with the majority being AAs, followed 
by 30-60 unit and 1 8-30 unit certificates. 

We cannot make causal claims that EWD helped 
students earn more degrees, but CTE deans 
reported that EWD funding had impacted student 
completion rates. Sixty-seven percent of the 
CTE deans said students were “very much” 
or “somewhat” more likely to complete their 
programs of study because of EWD funding. 

Transfer and employment outcomes 
were unknown 

We do not know how many EWD participants were 
either transfer-ready or transferred to a college, 
since the Perkins IV Core Indicator 12 data on 
these two outcomes were not available for the 
academic year we assessed. While transfer data 
were available on DataMart, we were unable to 
isolate our proxy for the EWD student population. 


Figure 2. Passing Rate for EWD Courses 
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Figure 3. 2013-2014 Associate Degrees Awarded 
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Since these data were unavailable, we also could 
not estimate how many student EWD participants 
entered employment. In their quarterly reports, 
grantees indicated that 1,135 people were placed 
in jobs. Furthermore, we did not have access 
to recent wage data collected by the CCCCO. 

The last reported year for wages earned by 
students who received a degree was 2008-2009, 
significantly before the implementation of SB 1402. 


Perceived Impacts on Employers 

Employers received a wide range of EWD 
services. Grantees reported in their quarterly 
reports that these services led to: 

• Hiring 1,498 employees. 

• Retaining 7,194 employees (i.e. 
employees not leaving). 

• Generating $75,430,844 in revenue. 

• Achieving $99,319,944 in sales. 

• Developing 75 new products. 

• Creating 94 new services. 


Figure 4. 2013-2014 EWD-Related Certificates Awarded 



Source: CCCCO MIS Data Mart 
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Most employers unfamiliar with EWD; 
however those that were familiar 
found EWD services useful 

Thirty-two percent of employers who participated 
in our survey were familiar with or had heard of 
EWD. Thirteen percent of respondents had either 
participated in the program or received services, 
and all of these employers said EWD provided 
the training needed by industries in their region. 

The majority of this subset believed EWD would 
increase their current employees’ skill sets, as well 
as provide services and programs responsive to 
their region’s workforce needs. They also generally 
found the programs they selected to be useful. 
Employers reported that one of the biggest 
barriers they encountered with EWD was not 
knowing enough about the program (40 percent). 
Furthermore, 36 percent of employers believed 
EWD services did not align with their business 
needs. However, they did identify the services they 
would find useful to receive, including training for 
their employees and aligning community college 
courses to their business needs (see Appendix A, 
Table A-9). This demonstrated that employers were 
unaware of what EWD provides, as those were 
two services EWD already offered to employers. 

Perceived impacts on education and 
workforce entities 

EWD grantees and stakeholders were generally 
positive about the perceived effects of EWD 
on community colleges, employers, and 
California’s workforce system. We found that: 

• Nearly half (47 percent) of DSNs believed EWD 
was meeting its goal of advancing California’s 
economic growth and global competitiveness 
“very well,” and 50 percent reported EWD 
was meeting its goal “somewhat well.” 

• Among CTE deans, 17 percent reported 
that EWD had been very effective in fulfilling 
its mission, and 44 percent reported 

EWD had been somewhat effective. 


• Of the regional consortia chairs, 29 
percent reported that EWD had been 
very effective in fulfilling its mission, 
and the remaining 71 percent stated 
that EWD was somewhat effective. 

• All (1 00 percent) of employers who participated 
in our survey and were aware of the EWD 
program said EWD was meeting its goal of 
advancing California’s economic growth and 
global competitiveness “somewhat well.” 

Analysis of EWD Strengths and 
Challenges 

Our analysis found that EWD had many strengths 
and several challenges. The strengths often 
stemmed from the program’s design. EWD 
also had challenges, likely related, in large 
part, to the program’s new structure. They 
may be minimized as the program matures. 

EWD Strengths 

Many of the strengths of the program stem 
from the design of its structure and focus 
on building relationships. According to 
grantees, EWD’s primary strengths were: 

• Improved employer-college connections 
through quick response to employers’ needs. 

• Strategic planning, a mindset of care and 
service, and more interaction with community 
college activities through DWM framework. 

• Fostering camaraderie between 
grantees and stakeholders to build 
economies and workforces. 

• Explicit focus on high growth sectors 
to better target job skill needs. 

• Constructing and increasing the visibility 
of career pathways in K-12. 

• A goal of continuous improvement 
through metrics and workshops. 
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Improved employer-college connections 
through quick response to employers’ needs 

Grantees noted that EWD was responsive to 
employer needs, both long term and short term. 
Since those needs are dynamic, grantees stressed 
that they must be able to move quickly to assist. 
While they frequently could not respond as fast 
as employers desired, grantees said they were 
more responsive to employers than were the 
community colleges, and therefore played a key 
role in connecting their sectors’ colleges with 
businesses seeking workers with the right skills. 

Strategic planning, a mindset of care and 
service, and more interaction with community 
college activities through DWM framework 

EWD grantees felt the Doing What Matters 
for Jobs and the Economy program, with its 
goal of closing the skills gap by preparing 
students to enter and stay in the job market, 
provided a much-needed strategic framework 
that helped them organize and prioritize 
their objectives, an emphasis on thoughtful 
planning, care and service for their regions, and 
encouragement to cast off ineffective practices. 
Grantees also noted that employers were more 
responsive to being involved in a project titled 
“Doing What Matters for Jobs and the Economy.” 
The grantees also approved of the shift in 
EWD’s emphasis from economic development, 
business growth, and incumbent workers to faculty 
development, curricula, and student success. 

Fostering camaraderie between 
grantees and stakeholders to build 
economies and workforces 

One of the biggest strengths noted about the 
EWD program was the people. Grantees and 
stakeholders alike appreciated the camaraderie 
fostered by EWD that was creating a partnership 
of individuals and groups working together 
to build regional economies and workforces. 

This focus on teamwork to improve outcomes 
was singled out as a driving force in EWD. 


Explicit focus on high growth sectors 
to better target job skill needs 

Grantees reported that EWD’s focus on key 
sectors to target and better understand the 
unique job skills those sectors require enabled 
them to reap greater returns for students, 
workers, and regional economies. They felt this 
approach, compared to placing a broad focus 
on CTE in its entirety, led to better outcomes. 

Constructing and increasing the 
visibility of career pathways in K-12 

Grantees said that EWD provided a plan 
that made connecting with K-12 students, 
especially those in middle and high schools, 
a priority. This plan included promoting career 
awareness to adolescents, reaching out to their 
teachers, counselors and principals, and other 
activities to build pipelines from K-12 to higher 
education and to the workplace. The linking 
of other grants, like SB 1070 and California 
Career Pathways Trust funds, further allowed 
EWD grantees to spend time coordinating with 
high schools about workforce development. 

Goal of continuous improvement 
through metrics and workshops 

Grantees underscored the importance of 
EWD’s focus on continuous improvement, saying 
the program’s structure was new and would 
continue to require fine tuning. They said they 
were dedicated to examining metrics and data to 
find ways to improve their work. Some grantees 
were holding workshops to further their skills. 

Areas of Challenge 

Most of the challenges we found can be traced 
to the newness of the EWD program structure 
and grantee roles. These challenges were: 

• Lack of clarity about DSNs’ roles 
and accountability, which impeded 
DSN-stakeholder relations. 

• Complicated funding and grant 
management, a barrier for grantees. 

• Difficult data collection logistics for grantees. 
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• Lack of employer and community college 
awareness about EWD activities and services. 

• Community colleges’ rigid structure, 
a barrier to grantees’ objectives. 

• A strict regional focus, complicating 
DSNs’ ability to adequately serve. 

• Frustration over the selection process 
for priority and emergent sectors. 

Lack of clarity about DSNs’ roles 
and accountability, which impeded 
DSN-stakeholder relations 

While praising the flexibility of the DSN role, 
grantees, regional consortia chairs, and CTE 
deans expressed concern about the lack of clarity 
about what DSNs were supposed to do and to 
whom they were responsible and accountable. 

As a result, stakeholders were unsure how to 
work with DSNs. CTE deans said this issue 
was one of the main challenges facing EWD. 

Hosting EWD grantees at colleges contributed 
to this confusion. While DSNs reported two 
major benefits to being located at a college— 62 
percent said they gained understanding about 
how colleges function and 59 percent of them 
said it enabled them to work better with their host 
colleges— several drawbacks existed. Almost 
a quarter of DSNs stated that being stationed at 
a host college caused the following drawbacks: 

• Confusion over to whom they report. 

• Limited ability to hire staff. 

• Requests by the host college to complete 
tasks outside the grant’s scope. 

A handful of DSNs expressed frustration over 
not being able to spend money on regional work, 
as their host colleges would only allow them 
to spend money on college-related activities 
there. CTE deans surveyed also consistently 
said they saw DSNs spending significant 
amounts of time serving their host colleges, to 
the detriment of other colleges in their regions. 


Difficult funding and grant management, 
a barrier for grantees 

EWD grantees expressed frustration over 
how grant funding was managed, citing: 

• Differences in policies and procedures 
for each community college that hosts or 
manages a grant or sub-grant contract. 

• Inconsistencies and errors in grant paperwork. 

• Inconsistencies in work plan approvals 
across grantees. 

• Different reporting requirements and processes 
for the variety of CCCCO-managed grants. 

• Late receipt of grant paperwork and funds. 

• Single-year grants that require 
re-application each year. 

EWD funds must be distributed by a host college 
and were overseen by the CCCCO. 13 Each 
host college had different processes for hiring, 
spending, and reporting, and grantees reported 
that CCCCO grant staff were inconsistent in 
their management of grants. Grantees also 
found it frustrating to receive funding through 
host colleges. This method forced them to 
maneuver through multiple processes— each 
school has its own— for receiving these funds. 
Moreover, when the grantee provided supports 
for another college in the region, the grantee 
had to go through the contracting process with 
two colleges - the host college plus the other 
college in the region that was receiving support. 

The CCCCO additionally had different reporting 
requirements for each grant, so EWD grantees 
who received multiple grants (like all DSNs who 
received both SB 1 402 and SB 1 070 grants) had 
to report on each separately. Grantees found the 
different reporting websites and processes for 
each grant to be frustrating and time-consuming. 

Combined with the missed deadlines experienced 
by those processing the grants and paperwork, 
grantees spent significant amounts of time dealing 
with the bureaucracy of contracts and grants, 
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rather than on providing services to support the 
EWD mission. Moreover, grantees reported that 
these delays, which sometimes lasted months, 
required them to scramble to spend their single- 
year grant funds before their contracts ended. 

They added that these year-long grants 
discouraged them from making larger, 
longer-term investments in economic 
and workforce development. 

Challenging data collection logistics 
for grantees 

Grantees noted the EWD program’s focus on data 
and metrics and said it underscored an emphasis 
on data-driven decision-making and accountability, 
but added that data limitations and lack of clarity 
about metrics made it difficult for grantees to 
understand what services should be provided 
and how EWD was performing. They reported: 

• Being asked to provide the same 
data to multiple data systems. 

• The requirement to input data 
they could not access. 

• Unclear definitions about variables in the data. 

• Data that often were inaccurate 
and inconsistent. 

• Metrics so numerous that it was difficult to 
understand which were most important. 

Grantees also said it was difficult to 
quantify the benefits and impacts of EWD, 
particularly around employment. 

In particular, DSNs said they sometimes had 
to submit the same data in quarterly reports to 
the CCCCO as well as to LaunchBoard. They 
reported other difficulties inputting data. They were 
concerned that they were being held responsible 
for inputting student-level data they could not 
access. Finally, DSNs said they were not given 
enough information about data definitions to 
ensure they were providing the correct data. 


Moreover, not only DSNs, but all grantees reported 
that LaunchBoard was not ready for use and 
was poorly introduced. They specifically noted 
concerns about the accuracy and consistency of 
the data they were trying to work with. Grantees 
also said the large number of metrics - including 
momentum points and leading indicators— were 
difficult to analyze, which made it difficult for 
them to decide where to focus their time. 

Lack of employer and community college 
awareness about EWD activities and services 

Employers surveyed noted a limited awareness 
among them about EWD programs and services. 
Further signaling a need to build program 
awareness was the fact that services the employers 
reported wanting to receive were similar to the 
ones the EWD program already provided. 

SNs and DSNs also reported that many community 
college faculty, staff, and administrators were not 
aware, in general, of EWD and of their specific 
roles as SNs and DSNs. This made grantees’ 
work on the campuses more difficult, and they 
spent significant time with people having to 
explain the EWD Program. They said many 
stakeholders were lukewarm about EWD’s role with 
community colleges since they hadn’t received 
communication about the program. Some 
DSNs noted that Doing What Matters for Jobs 
and the Economy messaging was excellent at 
the state level, but that it had not effectively 
reached the campuses or regional consortia. 

Community colleges’ rigid structure, 
a barrier to grantee objectives 

Community colleges are at the core of EWD, 
but grantees felt they were, at times, barriers 
to the program’s effectiveness. The issues 
cited were that community colleges: 

• Responded and made changes slowly. 

• Had faculty and staff unwilling or 
unable to revise courses. 
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• Were hampered by their rigid structure, 
processes, and bureaucracy. 

• Lacked an understanding of EWD. 

• Were hesitant to join the EWD program. 

• Operated autonomously, not 
under CCCCO’s direction. 

Of the DSNs surveyed, 34 percent of them reported 
barriers to coordinating with community colleges 
(see Table 6). Grantees indicated that the colleges 
were slow to respond and make changes. Their 
rigid structure, along with some faculty who 
were unwilling or unable to revise courses, made 
it difficult to make suggested EWD changes 
quickly. Grantees said this led employers to stop 
working with the community colleges and instead 
to turn to private colleges, which could more 
quickly revise curricula and offer new programs. 

As noted earlier in this report, grantees felt that 
a lack of awareness at community colleges of 
EWD inhibited the grantees’ ability to connect 
with those colleges. Grantees also said the 
colleges were concerned with their own individual 
agendas and not regional needs, and that they 
each operated independently, governed by their 
own boards of trustees and not by the CCCCO. 

Grantees felt the community colleges’ autonomy 
ultimately required them to take on greater 
responsibilities because the CCCCO could not 
require the colleges to act. For example, DSNs 
reported having to enter student-level data into 
LaunchBoard even though the colleges, not the 
DSNs, were the ones with access to these data. 

In seeking to overcome these challenges, several 
grantees said they built strong relationships with 
CTE deans who understood EWD goals, and 
that these deans were influential in facilitating 
change on their community college campuses. 
These changes included supporting curricular 
revisions, providing faculty professional 
development, and upgrading classroom 
technology. However, not all colleges had CTE 
deans who could impact campus processes. 


A strict, regional focus, complicating 
DSNs’ ability to adequately serve 

While grantees acknowledged the value of 
EWD’s regionally-oriented structure, they noted 
that the region’s geographic borders presented 
some challenges and may need refinement. 

This regional structure enabled the 
program to be more effective by: 

• Recognizing the different needs of regions. 

• Promoting coordination among 
stakeholders in the region. 

• Fostering innovation through the 
ability within a region to act quickly 
and test ideas on a small scale. 

The regional focus promoted coordination 
on shared goals and objectives among 
a region’s community colleges, employers, and 
stakeholders, and it also recognized that individual 
regions have different needs. Regional consortia 
chairs commonly attributed the success of their 
regions to high levels of collaboration, activity, 
and engagement. These structured collaborations 
often led to greater cohesiveness and cooperation 
among partners and improved results. 

Additionally, this regionalization allowed for new 
ideas to be tested on a smaller scale and for 
innovations to be launched more quickly. It also 
further enabled explicit connections to state and 
federal workforce programs that are regionally 
structured, such as Workforce Innovation and 
Opportunity programs and Workforce Investment 
Boards. By collaborating with other programs that 
shared the same goals, grantees furthered their 
impact on economic and workforce development. 
The result of these key connections was well- 
described by one DSN as the creation of a 
“regional capacity development program.” 

But this structure also presented weaknesses: 

• Different stakeholder groups, such as 
K-12 districts, had their own districts 
and boundaries that did not always 
fit easily into EWD regions. 
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Table 6. Percent of DSNs Experiencing Barriers in Providing Services 


Experienced 
with Colleges 

Barriers (n=37) 

Experienced 

with Businesses 

(n=38) 

There is not enough funding to provide services at the level of 
quality I’d like. 

47% 

37% 

There is not enough time to provide services at the level of 
quality I’d like. 

37% 

32% 

It has been hard to coordinate with them. 

34% 

16% 

1 have not experienced any barriers. 

18% 

32% 

They are spread too far apart geographically. 

18% 

13% 

They are not interested in receiving my assistance. 

18% 

5% 

1 do not have the necessary tools to provide services at the level 
of quality I’d like. 

5% 

18% 


Source: EWD Program Evaluation Survey of DSNs 


• Grantees found it challenging to serve 
vast regions and territories with a large 
number of community colleges. 

• DSNs often had to cross regional 
boundaries to additionally serve 
adjacent territories lacking a DSN in a 
particular priority or emergent sector. 

Grantees and CTE deans noted that the regional 
strategy needs refinement. Unlike cities, counties, 
and districts, regions are not officially designated. 
As such, regional boundaries may differ from entity 
to entity. DSNs serving regions designated by 
EWD may be faced with needs from stakeholders 
outside of that region. For example, when adjacent 
regions with shared economies selected different 
sectors, DSNs whose work covered one region but 
not the other were frequently called upon to assist. 

The size of regions was also highlighted as 
a barrier by grantees, CTE deans, and regional 
consortia chairs. They said large regions left 
grantees struggling to serve all of the colleges and 
programs in that region. In the CTE deans’ survey, 
72 percent of deans agreed this was a primary 
challenge facing EWD. Moreover, CTE deans 


consistently said they did not have interaction 
with many of the DSNs and did not feel the DSNs 
were spending equitable amounts of time with 
each college in a region. Regional consortia 
chairs said that a large geography made it difficult 
for key stakeholders who were not centrally 
located to convene and unite on shared issues. 

Confusion over the selection process 
for priority and emergent sectors 

EWD grantees reported concerns about the 
processes for selecting priority and emergent 
sectors, and the lack of EWD buy-in by 
stakeholders. These concerns were that: 

• The selection process was too rushed. 

• Employers were not at the table 
when selections were made. 

• There was a lack of transparency, and 
perceived inequality, in the CCCCO’s process 
of placing additional DSNs in regions. 

• Many regions did not use criteria 
or data in selecting sectors. 
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• Votes for sectors were based on self- 
interest rather than on promoting 
the region’s economic growth. 

• The 1 0 sector options were sub-optimal. 

• EWD should focus on all employer needs. 

Priority and emergent sectors were chosen 
at regional consortia meetings. The CCCCO 
provided suggestions on how regions should 
select sectors, but ultimately regional consortia 
made their own decisions about which sectors 
to select. 14 When the popular vote was taken 
without criteria, grantees reported that regional 
consortia members tended to vote in their own 
interests - for the sectors that best suited their 
own programs or for the larger programs at their 
campuses. Some grantees expressed concern 
that employers were not at the table to help make 
these sector selections. Regardless of the process 
used, grantees were troubled by the fast speed of 
the process. They felt there was little, if any, lead 
time to study the sectors and labor market needs. 

Grantees also worried that the number of priority 
and emergent sectors kept changing. In the 
end, grantees reported being unsatisfied with 
how many sectors each region had received. 

They said some regions needed an additional 
sector, but could not have one because they had 
reached the maximum of five sectors per region. 
While the EWD sectors were based on existing 
state sectors, grantees noted that not all sectors 
were industries. Examples they provided of such 
sectors were the small business and the global 
trade and logistics sectors. The structures and 
processes needed by such sectors differed 
from those that worked for industries and 
were more comparable to those required by 
other sectors in EWD regions. This situation 
prompted grantees to promote discussion about 
new ways to think about curricular alignment, 
career pathways, and workforce development, 
in general, in order to help find ways to better 
identify and serve all sectors in their territories. 


Grantees also stated that current sectors 
excluded public service sectors, like public 
safety and education. They reported that these 
exclusions seemed to be taken for granted, 
but that public sector occupations that met 
the same criteria as private sector occupations 
should be given equal consideration. 

Relatedly, 70 percent of the regional consortia 
chairs and 78 percent of the CTE deans who were 
surveyed felt the limited focus on priority and 
emergent sectors, rather than all major sectors, 
was a challenge facing EWD. Moreover, when 
CTE deans were asked what improvements they 
would recommend, 76 percent of them identified 
having DSNs for all industry sectors, not just 
sectors identified as priority or emergent. 
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Recommendations for EWD Program 
Improvement 


Across all participants we interviewed and 
surveyed, stakeholders generally believed the 
Economic and Workforce Development Program 
(EWD) has been successful in contributing to 
a highly skilled and productive workforce. The 
structure of the program under the Doing What 
Matters (DWM) framework is relatively new, and 
this is reflected in participants’ statements of 
the program’s effectiveness, impact, strengths, 
and weaknesses. While some of the issues 
highlighted will likely work themselves out as 
the program matures, we have identified several 
broad areas that should be targeted to improve the 
program’s success. Our recommendations include: 

• Clarify grantee roles and responsibilities, 
while maintaining a transparent and 
consistent system of accountability. 

• Refine funding and grant management 
to improve EWD efficiency. 

• Improve data and metrics used for EWD 
improvement and accountability. 

• Build awareness of EWD, particularly for 
employers and community colleges. 

• Increase community colleges’ ability to 
respond quickly to employer needs. 

• Refine the program’s regional focus to 
encourage greater collaboration. 

• Formalize a structure for sector 
selection that promotes a thoughtful 
and data-driven approach. 

Many of our recommendations align with those of 
the CCC Board of Governors’ (BOG) Task Force on 
Workforce, Job Creation and a Strong Economy (for 
a comparison of both sets of recommendations, 
see Appendix A, Table A-1 0). The CCCCO has 
begun addressing its recommendations. It is not 
certain, that ours, which are specific to EWD, 


will be addressed in the course of the BOG task 
force’s work, but we believe it is feasible that 
they could be acted upon at the same time. 

One aspect of grant management that we 
recommend is not included in the BOG task force 
recommendations: creating a single portal for all 
grants managed by the CCCCO. As such, we 
will discuss this recommendation here in greater 
detail than the others. We also will suggest specific 
actions for addressing challenges facing EWD. 

Clarify Grantee Roles and 
Responsibilities, while Maintaining 
a Transparent and Consistent System of 
Accountability 

Clarify EWD Grantee Roles, Accountability, 
Responsibility 

The CCCCO needs to make clear the role of the 
EWD grantees, especially the DSNs, including 
what they are responsible for and to whom they 
are accountable. The information needs to be 
communicated to EWD grantees, along with 
the stakeholders. Of particular importance is to 
define the relationship between grantees and 
their host colleges. Are host colleges merely 
fiscal agents for the grantees’ grant? Or do they 
have input on the grantees’ day-to-day activities 
beyond ensuring the grant contract is fulfilled? 
Moreover, the CCCCO must ensure that the host 
colleges are not overstepping their authority by 
requiring EWD grantees to perform tasks for them 
beyond the grantees’ contracted responsibilities. 
We recommend the CCCCO explore alternate 
structures, such as a joint powers authority, that 
free colleges from having to host EWD grantees. 
Additionally, the CCCCO currently depends 
on SNs to provide feedback on DSN services 
and on-the-ground activities, but DSNs report 
that SNs’ ability to understand and accurately 
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represent them and their activities varies. The 
CCCCO should make clear that the responsibility 
of SNs is to understand and accurately 
represent DSNs on statewide task forces. 

Decrease Discrepancies Between What DSNs 
Are Allowed to Do and Pay for 

In increasing clarity on grantee roles and 
responsibilities, the CCCCO and host colleges 
need to maintain clear, consistent, and transparent 
guidelines on allowable grant activities. This may 
require revamping the training and guidelines 
that grant analysts use to make decisions 
about approvals to ensure decisions are less 
susceptible to interpretation. The CCCCO and 
host colleges also should consider implementing 
an appeals process for grantees to address 
any inconsistencies identified in approvals. 

Improve Grant Accountability 

The CCCCO has many metrics in place for the 
EWD program, and contracts lay out grantees’ 
roles and responsibilities. However, some grantees 
expressed concern that not all grantees were 
spending grant funds as the EWD grant stipulates, 
yet there was no system or process in place to 
hold grantees accountable. We recommend the 
CCCCO investigate these claims and implement 
an accountability system that will capture these 
lapses and take corrective action to assist grantees 
in conforming to grant requirements. A possible 
solution could be the risk-based monitoring 
system the CCCCO uses for the Perkins grants. 
This system may alleviate some of the issues 
since it provides greater calibration and guidance 
and depends on a more organized scheme of 
critical flags that automatically trigger even 
further support and monitoring. We recommend 
the CCCCO consider a similar model for EWD. 


Refine Funding and Grant Management 
to Improve EWD Efficiency 

Provide Consistent Multi-Year Funding 

We recommend providing EWD grantees with 
multi-year funding to encourage long-term 
planning and projects. The year-by-year funding 
process is an ineffective method of providing 
funds for larger projects. EWD grantees reported 
this annual funding process was often a barrier 
to innovative, long-range projects. Moreover, 
it forced them to navigate bureaucratic grant 
contracting processes annually. In these 
processes, the grantees often faced delays and 
errors and had to manage them, taking time 
away from their delivery of grant services. 

Create a Single Portal for All Grants Managed 
by the CCCCO 

Grantees reported spending significant amounts 
of time uploading data and accountability 
reports to various systems for CCCCO grants. 

Not only are many of the metrics redundant, 
but grantees have to spend time learning and 
uploading similar reports to separate systems 
all managed by the CCCCO. We understand 
the CCCCO receives funds from different 
sources, yet we recommend investing time in 
understanding how the grant system could be 
simplified for grantees. We believe a change will 
also assist the CCCCO in managing grants. 

Improve Data and Metrics Used for EWD 
Improvement and Accountability 

Regional consortia chairs noted “improve data 
available to EWD grant recipients” as their top 
recommendation for EWD improvement (86 
percent). It will take a concerted effort from 
grantees, colleges, and the CCCCO to ensure 
data are not only current, but that data systems 
are easy to use and help to discern impacts. 
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Hone in on Fewer Key Metrics 

EWD uses 33 common metrics, which leave 
grantees overwhelmed about what is important 
and spending significant amounts of time 
trying to collect data on metrics beyond 
their control. We recommend EWD hone 
in on fewer metrics, but ensure they are at 
the core of the program’s mission. These 
metrics should be clear, measurable, and 
easily-collected performance measures. 


Align Accountability and Performance Metrics 
to Program Mission 

EWD’s mission is focused on the economy 
and workforce development, but many 
accountability and performance metrics 
concern student performance, which EWD 
grantees felt they had no control over and did 
not explicitly connect to the program mission. 

We recommend making clearer connections 
between EWD grantees and student performance 
and increasing alignment of accountability and 
performance metrics to the mission of EWD. 


Make Data Accessible to Grantees 

The CCCCO needs to make high quality data 
accessible to grantees. DSNs reported being 
responsible for inputting student level data 
into LaunchBoard, but not having access to 
the data they were being held responsible for 
inputting. Furthermore, EWD grantees stated 
being held responsible for student outcomes 
even though they did not have data they could 
analyze to understand their impact on students. 



Access to LaunchBoard Should Be Limited 
Until it is Ready for Prime Time 

Grantees were expected to use LaunchBoard 
before it was complete. Grantees’ poor experience 
with the incomplete system left them disinterested 
in using the system. We recommend completing 
the LaunchBoard system before having grantees 
enter data and begin using it. At that point, it must 
be ensured that grantees have proper training to 
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use the data, that it is clear who is responsible for 
entering data, and that data definitions exist to 
provide consistency across colleges and over time. 


Make More Training on Understanding and 
Using Labor Market Information Available 

While some grantees had strong research 
backgrounds, others desired more training on 
how to make use of labor market information. 
These grantees felt they had too little knowledge 
to even solicit the help of the Centers of 
Excellence. We recommend making more 
training available and making CoE technical 
assistance services more approachable. 


Make Clear the Role of CoE to Support DSNs 

Some DSNs were unclear about the specific 
services that CoE provided. We recommend CoE 
create a form that describes its different services so 
that CoE DSNs know how CoE can support them. 


Help Measure the Impact of the Programs 

Grantees felt left alone in figuring out how to 
measure the impact of their services. They wanted 
more support and guidance in understanding 
how this could be done. We recommend the 
CCCCO develop metrics and mechanisms 
that grantees can adapt for their own use in 
estimating the impact of their services. 

Build Awareness of EWD, Particularly for 
Employers and Community Colleges 

The CCCCO needs to lead the effort to build 
awareness of EWD. Senior officials must market 
the program to employers and community colleges 
and provide grantees with greater capacity 
and resources to enable them to market it, too. 
Marketing efforts to the colleges should focus 
on promoting the importance of career technical 
education and of economic and workforce 
development, in general, along with EWD, in 
particular. There should be more information 
shared about EWD staff roles and responsibilities, 
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especially among CTE deans, chief instructional 
officers, and chief executive officers. Increasing 
clarity on grantee roles will assist in this effort. 
Marketing to employers should focus on what 
EWD is and how it can help employers. We found 
that employers seek the types of services that 
EWD provides. Co-marketing EWD with other 
community college efforts in CTE, such as SB 
1 070-funded efforts, or with DWM efforts, may 
prove more effective than marketing EWD alone. 

Increase Community Colleges’ Ability to 
Respond Quickly to Employer Needs 

Work to Speed Up Curriculum Development 
and Review Processes 

A common complaint was how long it took to 
create a new course or program or to revise 
existing courses. The CCCCO could revise its own 
part of the curriculum and program review process 
and assist campuses and regions in streamlining 
their curriculum review and approval processes. 
Additionally, addressing issues concerning 
community college faculty and administrator 
awareness, understanding, and buy-in of EWD 
may help speed changes to curricula. Grantees 
noted that some faculty, because of their lack of 
EWD buy-in, were unwilling to work with SNs,, 
which made it difficult to revise courses quickly. 

Refine the Program’s Regional Focus to 
Encourage Greater Collaboration 

For Large Regions, Provide Enough DSNs to 
Adequately Meet Program Objectives 

Regions with large numbers of colleges should 
be reduced to a more manageable size or be 
provided with more DSNs. While macro regions 
have been broken into micro regions, DSNs are 
still responsible for supporting the needs of the 
macro region. It is not realistic for one or two DSNs 
to support nearly 30 community colleges’ needs. 

Similarly, in a region that is geographically large, 
a single DSN cannot easily visit all the community 
colleges and economic areas in that territory. 


Given that such a major part of the DSN’s role 
is connecting employers with colleges, it is 
essential that the DSN be able to spend significant 
amounts of time within each college’s community. 

If the region has distinct economic zones, we 
recommend splitting it into smaller regions 
with their own DSNs, In large regions without 
those zones, we recommend multiple DSNs be 
strategically located across the region. These 
DSNs would work together and separately to 
support local communities and the entire region. 

In both instances, we recommend the CCCCO 
map backward from the outcomes it seeks to 
determine what level of services and activities are 
needed. The CCCCO then should project how 
many more DSNs would be necessary, and how 
much additional funding is needed from the state 
for these extra grantees, or if resources could 
be allocated from elsewhere in the program. 

Factor Demands from Neighboring 
Regions into DSN Workloads 

DSNs who cover a sector that a neighboring 
region decided not to include as a priority or 
emergent sector were called upon to support that 
neighboring region’s economic and workforce 
development needs. Unless EWD decides to 
cover all sectors, we recommend that DSNs’ 
workloads be taken into account when considering 
these additional demands of their time. 

Allow a Program to be Developed and 

Offered by a District or Group of Colleges 
Instead of by Just One College 

EWD’s regional focus aims to break down silos and 
have community colleges coordinate with each 
other. At times, one college may have courses that 
another college does not, despite the two colleges 
being geographically close or offering courses 
online. The ability for colleges to incorporate 
each other’s courses into their programs would 
be an efficient way to promote student success 
and increase the number of certificates awarded. 


Aiming to Meet Workforce Needs: 
An Evaluation of the Economic and Workforce Development Program 


37 


This requirement, that a single college offer 
a program, is in the California Education Code 
and, thus must be revised by the legislature. 


the CCCCO could take a larger role by setting 
the criteria to be used, providing the labor market 
information to be considered, and determining the 
process by which the decision would be made. 


Formalize a Structure for Sector 
Selection that Promotes a Thoughtful 
and Data-Driven Approach 

Grantees, CTE deans, and regional consortia chairs 
expressed concerns about priority and emergent 
sectors. CTE deans and regional consortia chairs 
did not support the focus on a small number 
of sectors rather than on all sectors. Grantees 
reported that priority and emergent sectors were 
chosen in an inconsistent, interest-driven process. 

It is unclear if this is accurate, but it is clear that 
the sector selection process left many grantees 
and stakeholders dissatisfied. The CCCCO should 
formalize a structure for sector selection that 
promotes a thoughtful and data-driven approach. 
Furthermore, the CCCCO should ensure that 
regions understand and follow that process. 

Address Issues in the Sector 
Selection Process 

Grantees and stakeholders reported 
several challenges in the sector selection 
process. Commonly cited issues were: 

• Self-interest drives sector selection. 

• Labor market data were not consistently used. 

• Employers were not at the table. 

• Lack of clarity and constant changes in 
the number of sectors was allowed. 

• Public sector careers were excluded. 

The CCCCO should provide stronger guidelines, 
greater support, and at least some requirements 
for how priority and emergent sectors are 
identified and foster a clear decision-making 
process. Requirements could include the use of 
criteria and labor market information, along with 
a clear decision-making process. Alternatively, 
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Conclusion 


EWD is an important component of the 
state’s career technical education program. It 
connects employers with educators to promote 
alignment of workforce development needs and 
community college educational offerings. EWD fits 
into the umbrella DWM framework created by the 
CCCCO in an effort to motivate a coherent strategy 
in supporting California’s dynamic economy. 

EWD Program Design was Well Received as 
a Means of Structuring and Unifying Activities 
The structure of EWD helped to unify program 
processes, activities, and services to support 
economic and workforce development 
across the state, as reported by grantees and 
stakeholders. They consistently noted these 
key components of the program’s design: 

• Regional focus. 

• Multi-level delivery and support system. 

• Focus on coordinating with stakeholders. 

EWD’s regional focus acknowledges the diversity of 
local needs across the state while also supporting 
coordination between colleges and within local 
labor markets. The second component, the 
delivery and support system, recognized regional, 
state, sector-specific, and technical service 
needs. It also provided layered coverage of 
economic and workforce needs while supporting 
specialization and broader needs. EWD’s explicit 
focus on bringing together stakeholders with 
shared interests and its willingness to break 
down barriers to promote coordination was the 
third major design element of EWD that was 
well received. Together, these key components 
provide a foundation that EWD should utilize to 
improve and hone its processes and outcomes. 

Program Challenges Reflected 
the Realities of a Newly 
Restructured Program 

The challenges identified by grantees and 
stakeholders largely reflect the realities of 


a newly restructured program. The issues relate 
to the trials of implementation and include 
clarity in grantee roles, the management of 
grants, development of a new data system, and 
program awareness and marketing. A focus on 
continual program improvement, combined with 
resources to address these issues, will diminish 
these challenges. State investment in CTE, along 
with the CCC system’s attention to identifying 
and smoothing implementation issues, will 
greatly improve the program’s effectiveness. 

Most of our findings and recommendations 
reinforce previous recommendations regarding 
CTE, including our own at Edlnsights 15 and those 
of the CCC BOG Task Force on Workforce, Job 
Creation and a Strong Economy. The CCCCO 
begun acting on the BOG recommendations, 
and we expect that many of ours will be 
addressed in this process. However, some of 
our recommendations require support from 
the state legislature and from the colleges 
themselves, not just from the CCCCO. 

The limited scope of our evaluation provides 
a starting point for further analyses. We probed 
the perceptions of grantees in-depth, however, 
we could not build a rich understanding of the 
perceptions of many important EWD and CTE 
stakeholders, such as employers, faculty, and 
students. A sophisticated analysis of the impact 
and outcomes of EWD were beyond the scope of 
our work. This was due to both time constraints 
and the lack of student-level data. The unveiling 
of LaunchBoard 2.0 (targeted for 2017) could 
present opportunities for answering key questions 
regarding EWD outcomes and impact, such as: 

• What practices have the best outcomes, 
biggest impact, and greatest value-add? 

• Where are outcomes, impact, and value- 
add less than expected or demanded? 

• Where do investments need to be targeted? 
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Moreover, these questions could be answered 
by region, by statewide sector, and by region 
and sector combined. Since the revised 
version of LaunchBoard that is currently being 
created (“LaunchBoard 2.0”) aims to integrate 
efforts across the CCC system and from other 
stakeholders, further analyses using LaunchBoard 
2.0 data could aid understanding of how 
coordination and collaboration might promote 
student success and economic development. 

In our recommendations, we propose exploring 
new structures and means of managing the 
EWD program and other related CTE efforts. 

Further research on how structures like a joint 
powers authority can be used to more efficiently 
manage and administer grants would assist the 
CCCCO in executing its mission and achieving 
its goals. Supporting further research in these 
and other areas would allow the state, the 
CCCCO, and the CCC system to make better 
decisions based on data and evidence. 

Career technical education, including economic 
and workforce development, has been a growing 
priority in California. The state has increased 
investments in K-12, higher education, and 
workforce agencies. The CCC system has 
promoted the importance of CTE. The CCCCO 
has crafted the Doing What Matters for Jobs and 
the Economy strategy to align and make sense 
of numerous grants to assist CTE in a linked and 
concerted effort to support the state’s economy. 
Positive changes to EWD structure, its role in DWM, 
and the increasing importance of CTE demonstrate 
a growing promise to support students and 
workers in growing California’s economy. 
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Appendices 


Appendix A. Tables and Figures 

Table A-1. Services, People Served, and Hours by Grantee Type 


Grantee Type 

# of 

Services 

Service/ 

Grantee 

Services/ 

Grantee/ 

Quarter 

# of 

People 

Served 

People/ 

Grantee 

# of 

Hours 

Hours/ 

Grantee 

IDRC (n=12) 

168 

14 

2.3 

1,688 

141 

2,988 

249 

CoE (n=7) 

26 

4 

0.7 

0 

0 

0 

0 

SN (1 n=0) 

335 

34 

5.7 

3,540 

354 

22,604 

2,260 

DSN (n=66) 

2,044 

31 

5.2 

55,235 

837 

265,713 

4,026 

Contract Ed (n=1) 

4 

4 

0.7 

19,275 

19,275 

5,597 

5,597 

Weighted Avg 

1,463 

27 

4.5 

38,755 

831 

185,464 

3,093 

Total 

2,577 



79,738 


296,902 



Source: 2013-2014 Grantee Quarterly Reports. 
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Table A-2. Number of Overall Services Provided per Grantee by Sector 
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Manufacturing 

& Advanced 
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3.0 

2.6 

0.6 

2.6 

4.6 

1.3 

Advanced 

Transportation & 
Renewable Energy 

3.3 

1.0 

2.8 

1.5 

3.8 

4.0 

3.0 

Agriculture, Water 

& Environmental 

Technology 

3.2 

2.6 

3.4 

2.6 

4.0 

3.4 
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& Utilities 
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1.3 

3.0 

2.7 

4.0 
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Global Trade & 
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2.2 

0.5 

1.5 

1.3 

3.0 

3.1 

1.5 

Health 

2.7 

1.5 

1.8 

1.7 

4.5 

4.4 

2.1 

Info & 
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Technologies/ 

Digital Media 

3.4 

1.5 

2.8 

2.7 

3.1 

3.4 

2.1 

Life Sciences / 
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4.0 

0.5 

1.5 

3.5 

2.5 

4.0 

2.5 

Retail/ Hospitality/ 

Tourism 

2.3 

2.3 

0.7 

0.3 

3.0 

4.3 

3.3 

Small Business 

2.8 

1.2 

2.1 

1.7 

2.7 

3.0 

2.1 

Annual Average 

23.1 

12.2 

17.9 

14.4 

27.0 

30.1 

16.1 

Quarterly Average 

3.8 

2.0 

3.0 

2.4 

4.5 

5.0 

2.7 


Source: 2013-2014 Grantee Quarterly Reports. 
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Table A-3. SN Total Services to Colleges by Sector - Number of Times SNs Provided Each Service 
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0 

0 

2 

4 

4 

12 

Advanced 
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Renewable Energy 

3 

3 

2 

0 

3 

3 

2 

16 

Agriculture, Water 

& Environmental 

Technology 

5 

1 

2 

4 

5 

5 

0 

22 

Energy (Efficiency) 

& Utilities 

2 

1 

0 

0 

3 

4 

0 

10 

Global Trade & 

Logistics 

5 

2 

0 

2 

5 

5 

5 

24 

Health 

5 

0 

5 

5 

5 

5 

5 

30 

Info & 

Communication 

Technologies/ 

Digital Media 

4 

2 

1 

3 

4 

4 

3 

21 
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Biotechnology 

5 

0 

2 

2 

4 

5 

1 

19 

Retail/Hospitality/ 

Tourism 

4 

0 

2 

2 

4 

5 

1 

18 

Small Business 

4 

2 

3 

2 

3 

3 

2 

19 

Total 

37 

13 

17 

20 

38 

43 

23 

191 

Range 

5 

3 

5 

5 

3 

2 

5 

20 


Source: 2013-2014 Grantee Quarterly Reports. 
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Table A-4. DSN Average Number of Services Provided to Businesses by Sector 
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Needs Assessment 

Small Business 

Creation and/or 

Exporting Modules 

Professional 

Development for 

Workers 

Connect with Colle 

and Education 

Research 

Advanced 

Manufacturing & 
Advanced Technology 

2.8 

1.9 

3.1 

1.0 

2.6 

3.9 

0.3 

Advanced 

Transportation & 
Renewable Energy 

2.0 

2.8 

2.0 

0.3 

3.8 

3.8 

1.3 

Agriculture, Water 

& Environmental 

Technology 

1.2 

2.8 

2.4 

0.0 

1.8 

2.6 

1.4 

Energy (Efficiency) 

& Utilities 

1.7 

3.0 

2.0 

0.0 

1.0 

4.0 

0.7 

Global Trade & 

Logistics 

1.6 

2.5 

2.4 

1.5 

1.6 

2.6 

2.2 

Health 

1.5 

3.2 

2.6 

0.1 

3.6 

4.7 

1.5 

Info & Communication 

Technologies/ 

Digital Media 

0.7 

1.7 

1.8 

0.1 

1.0 

3.4 

1.0 

Life Sciences / 

Biotechnology 

0.0 

1.5 

0.5 

0.0 

2.5 

2.0 

0.0 

Retail/Hospitality/ 

Tourism 

2.3 

2.3 

3.0 

0.0 

2.0 

3.7 

2.3 

Small Business 

1.0 

1.5 

2.7 

2.1 

0.8 

2.7 

2.4 

Weighted Average 

11.7 

18.2 

20.4 

7.4 

15.6 

28.0 

12.9 

Quarterly Average 

2.0 

3.0 

3.4 

1.2 

2.6 

4.7 

2.2 


Source: 2013-2014 Grantee Quarterly Reports. 
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Table A-5. SN Total Number of Services Provided to Businesses by Sector 
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Advanced 
Manufacturing 
& Advanced 
Technology 


16 


Advanced 
Transportation & 
Renewable Energy 


15 


Agriculture, Water 
& Environmental 
Technology 


Energy (Efficiency) 
& Utilities 


20 


Global Trade & 
Logistics 


16 


Health 


Info & 

Communication 
Technologies/ 
Digital Media 


14 


Life Sciences / 
Biotechnology 


12 


Retail/Hospitality/ 

Tourism 


19 


Small Business 


20 


Total 


16 


29 


25 


16 


34 


15 


144 


Range 


14 


Source: 2013-2014 Grantee Quarterly Reports. 
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Table A-6. DSN Share of Services Provided to Colleges and Businesses by Macro Region 


Macro Region 

# of DSNs 

Total 

% of 

Service per 

Services 

Services 

DSN 

A (North Coast/lnland and Greater Sac) 

11 

276 

14% 

25 

B (Bay Area and Monterey/Santa Cruz) 

14 

249 

12% 

36 

C (Mother Lode and Central Valley) 

7 

142 

7% 

20 

D (South Central Coast) 

6 

220 

11% 

37 

E (San Diego/Imperial) 

7 

228 

11% 

33 

F (Inland Empire) 

7 

492 

24% 

35 

G (LA and Orange County) 

14 

437 

21% 

31 


Source: 2013-2014 Grantee Quarterly Reports. 

Table A-7. EWD Course FTES 


Sector 

2010/2011 

2011/2012 

2012/2013 

2013-2014 

2014/2015 

Advanced Manufacturing & 
Advanced Technology 

26,918 

25,243 

24,206 

24,558 

24,569 

Advanced Transportation & 
Renewable energy 

14,360 

13,833 

13,353 

13,486 

13,485 

Agriculture, Water & 
Environmental Technology 

9,564 

8,915 

8,514 

8,556 

8,415 

Energy (Efficiency) & Utilities 

2,790 

2,863 

2,622 

2,686 

2,423 

Global Trade & Logistics 

601 

628 

644 

590 

520 

Health 

55,219 

51,726 

48,970 

49,267 

48,734 

Info & Communication 

Technologies / Digital Media 

69,026 

63,100 

58,194 

57,423 

55,835 

Life Sciences / Biotechnology 

10 

10 

8 

8 

9 

Retail / Hospitality / Tourism 

22,586 

20,607 

18,778 

18,618 

18,009 

Small Business 

76,699 

69,866 

67,223 

69,588 

70,311 

Total 

277,772 

256,789 

242,510 

244,780 

242,312 


Source: CCCCO MIS Data Mart. 



46 


Education Insights Center at California State University, Sacramento 


Table A-8. EWD FTES Change 


Sector 

2010 to 2015 

Percent Change 

Advanced Manufacturing & Advanced Technology 

-2,349 

9% 

Advanced Transportation & Renewable Energy 

-8745 

6% 

Agriculture, Water & Environmental Technologies 

-1,148 

12% 

Energy (Efficiency) & Utilities 

-367 

13% 

Global Trade & Logistics 

-81 

13% 

Health 

-6,486 

12% 

Info & Communication Technologies / Digital Media 

-13,191 

19% 

Life Sciences / Biotechnology 

-1 

7% 

Retail/Hospitality/Tourism 

-4,577 

20% 

Small Business 

-6,388 

8% 

Total 

-35,461 

13% 


Source: CCCCO MIS Data Mart. 


Table A-9. Employer Responses to “What services would be useful?” 


What services would you find useful for your business or industry? 

Help for my business with training for my workers. 

53% 

Help align courses offered at the community colleges with my business needs. 

50% 

Provide data and research to help improve my business. 

30% 

Conduct an assessment of my business needs. 

17% 

Conduct an assessment of my technology needs. 

13% 

There are no services that 1 would find useful. 

13% 

Conduct an assessment of my manufacturing process capability. 

10% 


Source: EWD Program Evaluation Survey of Employers (n=30). 
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Table A-10. Comparison of EWD Program Recommendations to BOG Task Force Recommendations 


EWD Program Evaluation BOG Task Force Recommendations 

Recommendations 

Clarify grantee roles and 
responsibilities, while maintaining a 
transparent and consistent system of 
accountability 

17. Strengthen communication, coordination, and decision- 
making between regional CTE efforts and the colleges to 
meet regional labor market needs. 

Clarify EWD grantee roles, accountability, 
responsibility 

17a. Clarify the role and fiscal management structure of 
the regional consortia, sector navigators, deputy sector 
navigators, and technical assistance providers and their 
relationships with the CCCCO and the colleges. 

Decrease discrepancies between what 
DSNs are allowed to do and pay for 

Improve grant accountability 

Refine funding and grant 
management to improve 

EWD efficiency 

See below for specific recommendations 

Provide consistent multi-year funding 

22. Establish a sustained funding source to increase 
community colleges’ capacity to create, adapt, and 
maintain quality CTE courses and programs that are 
responsive to regional labor market needs. 

23. Create a predictable, targeted, and sustained funding 
stream that leverages multiple local, state, and federal 

CTE and workforce funds to support an infrastructure 

for collaboration at the state, regional, and local levels; 
establish regional funding of program start-up and 
innovation; and develop other coordination activities. 

Create a single portal for all grants 
managed by the CCCCO 

Not covered by any BOG task force recommendations 

Improve data and metrics used for 

EWD improvement and accountability 

4a. Develop, streamline, and align common outcome 
metrics for all state-funded CTE programs and ensure that 
they are compatible with federal reporting requirements. 

5b. Explore barriers, both real and perceived, to sharing 
data, and create new incentives for the timely sharing of 

data. 

6a. Provide labor market, workforce outcome, and student 
demographic data/information that are easily accessible 

and usable. 

Hone in on fewer key metrics 

Align accountability and performance 
metrics to program mission 

Make data accessible to grantees 

Access to Launch Board should be 

limited until it is ready for prime time 

6c. Provide technical assistance, data visualization tools, 
and analysis tools to colleges for the use of labor market 

and student outcome data. 



48 


Education Insights Center at California State University, Sacramento 


EWD Program Evaluation 

Recommendations 

BOG Task Force Recommendations 

Make more training on understanding 
and using labor market information 

available 

6c. Provide technical assistance, data visualization tools, 
and analysis tools to colleges for the use of labor market 

and student outcome data. 

Make clear the role of CoE to support 

DSNs 

6a. Provide labor market, workforce outcome, and student 
demographic data/information that are easily accessible 

and usable. 

Help measure the impact of 
the programs 

6. Improve the quality, accessibility, and utility of student 
outcome and labor market data to support students, 
educators, colleges, regions, employers, local workforce 
investment boards, and the state in CTE program 
development and improvement efforts. 

9. Improve program review, evaluation, and revision 
processes to ensure program relevance to students, 
business, and industry as reflected in labor market data. 

Build awareness of EWD, 
particularly for employers and 
community colleges 

21 . Create a sustained, public outreach campaign to 
industry, high school students, counselors, parents, 
faculty, staff, and the community at large to promote 
career development and attainment and the value of career 

technical education. 

Increase community colleges’ ability 
to respond quickly to employer needs 


Work to speed up curriculum 
development and review process 

8. Evaluate, revise, and resource the local, regional, and 
statewide CTE curriculum approval process to ensure 
timely, responsive, and streamlined curriculum approval. 

Refine the program’s regional focus to 
encourage greater collaboration 

17b. Ensure that the CTE regional framework is designed 
to do the following: 

1) Designate labor market-driven priority and emergent 
sectors in coordination with employers, workforce boards, 
and economic development entities. 

2) Coordinate colleges within the region to meet business 
and industry needs. 

3) Convene discussions about development of common 

CTE entry pathways and industry-valued credentials based 
on regional industry needs. 

4) Share best practices on regional coordination, 
communication, and decision-making. 

5) Conduct joint marketing, and facilitate asset and 
equipment sharing. 

6) Support joint professional development of faculty to 
respond to evolving skill needs of industry sectors. 

7) Provide other needs and strategies as prioritized by 
the region. 
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EWD Program Evaluation 

Recommendations 

BOG Task Force Recommendations 

For large regions, provide enough DSNs 
to adequately meet program objectives 

23. Create a predictable, targeted, and sustained funding 
stream that leverages multiple local, state, and federal 

CTE and workforce funds to support an infrastructure 
for collaboration at the state, regional, and local levels; 
establish regional funding of program start-up and 
innovation; and develop other coordination activities. 

Factor demands by neighboring regions 

into DSN workloads 

23. Create a predictable, targeted, and sustained funding 
stream that leverages multiple local, state, and federal 

CTE and workforce funds to support an infrastructure 
for collaboration at the state, regional, and local levels; 
establish regional funding of program start-up and 
innovation; and develop other coordination activities. 

17a. Clarify the role and fiscal management structure of 
the regional consortia, sector navigators, deputy sector 
navigators, and technical assistance providers and their 
relationships with the CCCCO and the colleges. 

Allow a program to be developed and 
offered by a district or group of colleges 
instead of by just one college 

10. Facilitate curricular portability across institutions. 

1 8. Clarify and modify, as appropriate, state regulations 
to allow colleges to regionalize course articulation along 
career pathways utilizing regional or state curriculum 

models. 

Formalize a structure for sector 

selection that promotes a thoughtful 
and data-driven approach 

7a. Create consistent mechanisms for improved regional 
engagement of business and industry in the curriculum 
development process. 

Address issues in the sector selection 


process 
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Table A-11. 2013-2014 Certificates Awarded 


60+ units 

30 to < 60 

units 

1 8 to < 30 

units 

12 to < 18 

units 

Total 

Advanced Manufacturing and Advanced Technology 

43 

1,600 

1,479 

147 

3,269 

Advanced Transportation & Renewable energy 

25 

1,134 

1,195 

361 

2,715 

Agriculture, Water & Environmental Technology 

36 

242 

635 

38 

951 

Energy (Efficiency) & Utilities 

0 

357 

457 

19 

833 

Global Trade & Logistics 

3 

16 

68 

3 

90 

Health 

719 

2,469 

1,035 

229 

4,452 

Info & Communication Technologies / Digital Media 

0 

1,296 

1,337 

476 

3,109 

Life Sciences / Biotechnology 

3 

7 

3 

18 

31 

Retail / Hospitality / Tourism 

61 

1,479 

835 

45 

2,420 

Small Business 

14 

2,695 

3,362 

647 

6,718 


Source: CCCCO MIS Data Mart. 


Table A-12. Number of Students Placed in Jobs and in Internships by Grantee Type 


Grantee Type 

# of Students placed in Jobs 

# of Students in Internships 

Contract Ed 

0 

0 

DSN 

1,090 

2,415 

IDRC 

21 

79 

SN 

24 

156 

CoE 

0 

0 

Total 

1,135 

2,650 


Source: 2013-2014 Grantee Quarterly Reports. 


Table A-13. Number of Employees Hired and Retained by Grantee Type 


Grantee Type 

# of Employees Hired 

# of Employees Retained 

Contract Ed 

0 

0 

DSN 

1,454 

6,561 

IDRC 

24 

146 

SN 

20 

487 

CoE 

0 

0 

Total 

1,498 

7,194 


Source: 2013-2014 Grantee Quarterly Reports. 
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Table A-14. Amount of Revenue Generated and Increased Sales by Grantee Type 


Grantee Type 

Revenue Generated 

Increased Sales 

Contract Ed 

$0 

$0 

DSN 

$75,068,044 

$64,716,231 

IDRC 

$0 

$0 

SN 

$362,800 

$34,603,713 

CoE 

$0 

$0 

Total 

$75,430,844 

$99,319,944 


Source: 2013-2014 Grantee Quarterly Reports. 


Table A-15. Number of Products and Services Developed by Grantee Type 


Grantee Type 

# of Products Developed 

# of Services Developed 

Contract Ed 

0 

0 

DSN 

59 

76 

IDRC 

6 

5 

SN 

10 

13 

CoE 

0 

0 

Total 

75 

94 


Source: 2013-2014 Grantee Quarterly Reports. 


Table A-16. Sectors with Highest Average Number of Service to Colleges 


Develop & Align Curriculum 

Life Sciences/Biotech (4.0) 

Curriculum Alignment 

Advanced Manufacturing (2.9) 

Certificate & Program of Study Development 

Agriculture, Water & Environmental (3.2) 

Career Curriculum Articulation 

Small Business (12.0) 

Faculty Professional Development 

Health (4.6) 

Connect to Businesses and/or Industries 

Small Business (12.0) 

Research & Studies 

Advanced Transportation and Renewables (2.8) 


Source: 2013-2014 Grantee Quarterly Reports. 
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Table A-17. Sectors with Highest Average Number of Service to Employers 


Develop Curriculum for Business 

Advanced Manufacturing (2.4) 

Alignment of Sector with Education 

Energy and Utilities (3.5) 

Business Needs Assessment 

Advanced Manufacturing (3.1) 

Small Business Creation and/or Exporting 

Modules 

Small Business (2.3) 

Professional Development for Workers 

Advanced Transportation and Renewables (3.4) 

Connect with Colleges and Education 

Health (4.4) 

Research 

Retail/Hospitality/Tourism (2.8) 


Source: 2013-2014 Grantee Quarterly Reports. 
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Table A-18. DSN Share of Services to Colleges and Businesses by Sector 


Sector 

#of 

DSNs 

Total 

Services 

% of 

Services 

Year 

Average 

Quarterly 

Average 

SERVICES TO COLLEGES 

Advanced Manufacturing & 

Advanced Technology 

8 

138 

12% 

17.3 

2.9 

Advanced Transportation & 

Renewable Energy 

4 

77 

7% 

19.3 

3.2 

Agriculture, Water & 

Environmental Technology 

5 

107 

9% 

21.4 

3.6 

Energy (Efficiency) & Utilities 

3 

53 

5% 

17.7 

2.9 

Global Trade & Logistics 

10 

131 

11% 

13.1 

2.2 

Health 

10 

187 

16% 

18.7 

3.1 

Info & Communication Technologies / 
Digital Media 

10 

190 

17% 

19.0 

3.2 

Life Sciences / Biotechnology 

2 

37 

3% 

18.5 

3.1 

Retail / Hospitality / Tourism 

3 

49 

4% 

16.3 

2.7 

Small Business 

11 

172 

15% 

15.6 

2.6 

SERVICES TO BUSINESSES 

Advanced Manufacturing & 

Advanced Technology 

8 

124 

14% 

15.5 

2.6 

Advanced Transportation & 

Renewable Energy 

4 

63 

7% 

15.8 

2.6 

Agriculture, Water & 

Environmental Technology 

5 

61 

7% 

12.2 

2.0 

Energy (Efficiency) & Utilities 

3 

37 

4% 

12.3 

2.1 

Global Trade & Logistics 

10 

144 

16% 

14.4 

2.4 

Health 

10 

172 

19% 

17.2 

2.9 

Info & Communication Technologies / 
Digital Media 

10 

97 

11% 

9.7 

1.6 

Life Sciences / Biotechnology 

2 

13 

1% 

6.5 

1.1 

Retail / Hospitality / Tourism 

3 

47 

5% 

15.7 

2.6 

Small Business 

11 

145 

16% 

13.2 

2.2 


Source: 2013-2014 Grantee Quarterly Reports. 
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Table A-19. DSN Outcomes and Impacts 


Outcomes and Impacts Sector with Weighted Quarterly 

Highest Average Average Average 

(all sectors) (all sectors) 

# of Businesses Served 

Small Business (450.3) 

1,705.9 

284.3 

# of Students Served 

Health (1 ,648.2) 

5,341.6 

890.3 

# of Employees Served 

Health (51 1 .3) 

2,041.1 

340.2 

# of Workshops/Trainings 

Advanced Manufacturing 
& Advanced Technology 
(1,771.5) 

3,835.4 

639.2 

# of Hours for Contract Education 

Advanced Manufacturing 
& Advanced Technology 
(12,896.4) 

13,153.3 

2,192.2 

# of Hours for Performance 

Improvement Training 

Advanced Manufacturing 
& Advanced Technology 
(3,253.1) 

5,420.6 

903.4 

# of Hours of Credit/ 

Non-credit Instruction 

Advanced Manufacturing 
& Advanced Technology 
(13,102.5) 

13,349.9 

2,225.0 

# Placed in Jobs 

Advanced Manufacturing 
& Advanced Technology 
(47.9) 

130.73 

21.8 

# Participated in Work-based learning 

Agriculture, Water & 
Environmental Technology 
(150.6) 

238.98 

39.8 

# of People Hired 

Small Business (81 .3) 

224.6 

37.4 

# of Employees Retained 

Advanced Manufacturing 
& Advanced Technology 
(262.3) 

833.3 

138.9 


Source: 2013-2014 Grantee Quarterly Reports. 
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Table A-20. SN Outcomes 


Qtrly 

Sector with Highest Average 

Outcomes and Impacts „ Total , , Average 

Share (all sectors) 

(all sectors) 

# of Businesses Served 

Health (49%) 

1,547 

154.7 

38.7 

# of Students Served 

Agriculture, Water 

& Environmental 

Technology (49%) 

452 

45.2 

11.3 

# of Employees Served 

Heath (82%) 

3,088 

308.8 

77.2 

# of Workshops/Trainings 

Health (96%) 

2,653 

265.3 

66.3 

# of Hours for Contract Education 

Life Sciences / 

Biotechnology (100%) 

48 

4.8 

1.2 

# of Hours for Performance 

Improvement Training 

Health (99%) 

20,724 

2,072.4 

518.1 

# of Hours of Credit/ 

Non-credit Instruction 

Life Sciences / 

Biotechnology (100%) 

1832 

183.2 

45.8 

# Placed in Jobs 

Life Sciences / 

Biotechnology (100%) 

24 

2.4 

0.6 

# Participated in Work-based 
learning 

Life Sciences / 

Biotechnology (99%) 

155 

15.5 

3.9 

# of People Hired 

Life Sciences / 

Biotechnology (100%) 

20 

2.0 

0.5 

# of Employees Retained 

Heath (97%) 

487 

48.7 

12.2 


Source: 2013-2014 Grantee Quarterly Reports. 
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Figure A-1. Percent of Services Provided by Grantee Type 



Source: 2013-2014 Grantee Quarterly Reports. 


Figure A-2. Percent of People Served by Grantee Type 



Source: 2013-2014 Grantee Quarterly Reports. 
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Figure A-3. Percent of People Served by Grantee Type 



Source: 2013-2014 Grantee Quarterly Reports. 
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Appendix B. Description of Evaluation 
Requirements 

Objectives of the Evaluation for the 
Legislature and the CCCCO 

The reauthorization of EWD (SB 1402) required the 
CCCCO to contract for an independent evaluation 
of the program (Chapter 361 , Section 2, Part 
52.5, Chapter 7. 88650.5). The CCCCO sought an 
evaluator to address a specific list of questions 
for use by the legislature in considering future 
funding for the program and by the CCCCO staff 
in making improvements to the EWD Program (see 
Appendix C for specific questions). The objectives 
outlined in the CCCCO solicitation were: 

1 . Assess the effectiveness of the totality of grant 
projects funded through EWD. 

2. Assess EWD’s contributions to the Doing What 
Matters for Jobs and the Economy framework. 

3. Investigate challenges and promising practices 
of serving K-12 schools, community colleges, 
industry, students, and workers through EWD 
and its grantees. 

4. Document and assess outcomes achieved 
through EWD. 

5. Gauge the effectiveness of the program in 
achieving the intent of SB 1402, Lieu. 

6. Produce an evaluation report suitable for 
submission to the legislature. 

The Education Insights Center (Edlnsights) at 
Sacramento State University was awarded the 
contract for the evaluation. Working with the CCCCO, 
Edlnsights designed an evaluation study to meet the 
purpose and objectives of the Request for Proposals 
from the CCCCO, given the short timeframe of 
the study and the limitations of available data. 

Data Collection and Analysis 

The objectives of this evaluation were to conduct 
a process study (examining perceptions, experiences, 
efforts, and roles of EWD grantees) and an outcomes 
study (reporting counts on who has participated 


in program services and activities), leading to an 
assessment of the program (overall evaluation of 
EWD’s impact on community colleges, industry, and 
the workforce systems) to provide the legislature and 
CCCCO with an understanding of the functioning of 
EWD. The evaluation combined interview, survey, 
and administrative data on students, grantees, 
employers, and other EWD stakeholders to develop 
a robust picture of the program and its outcomes. 

Overview of evaluation methods, data, 
and limitations 

The evaluation used a mixed-methods approach, 
incorporating both qualitative and quantitative data. 
The process study examined the roles of EWD 
grantees, including such issues as their experiences 
coordinating stakeholders and with data collection 
and use. The outcomes study was intended to 
capture the impact of EWD funding on student and 
employer outcomes. However, a traditional outcomes 
study in which participant outcomes are compared 
to a control group of non-participants was not 
feasible, given that EWD’s design did not include 
a control group. Since EWD grantees tended to braid 
several sources of funding to support CTE programs, 
and since EWD funds did not flow directly to CTE 
programs, there was no way to isolate the effects 
of EWD funding on students or programs, and there 
was no way to compare students who benefit from 
EWD funds versus those who do not. Therefore, it 
was not possible to use control groups or quasi- 
experimental design. Instead, we analyzed descriptive 
statistics (from CCCCO Management Information 
System data) and stakeholders’ perceptions about 
key program-related issues and outcomes. 

To best understand EWD, we used multiple sources 
of data and multiple methods to analyze the EWD 
program and assess stakeholders’ perceptions of 
it. For the process study, we used qualitative (e.g., 
semi-structured interviews) and quantitative (e.g., 
surveys) methods to answer questions about the roles 
and contributions of EWD grantees (e.g., SNs, DSNs, 
TAPs, and IDRCs) under the Doing What Matters 
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framework; coordination efforts among the grantees 
and with stakeholders; and experiences with data 
collection and use among grantees. We conducted 
interviews and/or surveyed all EWD grantees. 

For the outcomes study, we utilized qualitative (e.g., 
semi-structured interviews) and quantitative (e.g., 
surveys, administrative data, reports) methods to 
answer questions about perceptions of the overall 
effectiveness of EWD — its strengths and weaknesses, 
use of services and outcomes, and recommendations 
for improvement. Due to the design of the program, 
we were limited in the types of analysis we could 
conduct. EWD services have been linked with other 
services with shared objectives, so it was impossible 
to disentangle the outcomes of one program from 
another. EWD programs aimed to support all students 
in a sector and region, which left us without a control 
group that we could use to compare the outcomes 
of EWD students. Also, since these grants have 
been in place in some form since 1991 , we could 
not rigorously compare student outcomes prior to 
EWD to outcomes with the EWD program in place. 

As such, this left us without the ability to make 
causal inferences about EWD’s impact or outcomes. 
However, after receiving access to data from the 
CCCCO (quarterly expenditure and progress reports 
and student MIS data), we were able to report limited 
descriptive statistics on student characteristics, 
enrollment in CTE programs in EWD-related sectors, 
and completion numbers (such as program awards). 
Also, we captured grantees’ perceptions of the overall 
effectiveness and impact of EWD funds on student 
outcomes— and the perceptions of colleges using the 
services and resources provided by the grantees— 
through interviews and surveys of grantees, CTE 
deans, regional consortia chairs, and employers. 

We transcribed the interviews and used content 
analyses to uncover dominant themes to inform the 
program evaluation. We also analyzed the survey 
content data to examine patterns in the data. 


Data sources 

To build as accurate an understanding of EWD 
as possible, we used both quantitative and 
qualitative data and analytical methods on grantees, 
students, employers, and other stakeholders. 

Qualitative data and analysis 
The qualitative data and analysis were based on 
45 total interviews with all classifications of EWD 
grantees, and qualitative responses to surveys 
administered to DSNs, regional consortia chairs, 

CTE deans, and employers. We solicited interviews 
with all sector navigators, IDRC directors, and 
TAPs (except the contract education technical 
assistance providers, as we were not able to get 
their contact information). We solicited interviews 
with a sample of 30 out of 66 DSNs. The sample 
was based on regional and sector coverage to 
increase the generalizability of our interview findings. 
We received responses from and interviewed all 10 
SNs, three IDRC directors, and all TAPs solicited. Of 
the 30 DSNs solicited, we interviewed the 25 that 
volunteered to participate. As such, we conducted 
a total of 45 interviews (1 0 SNs, 25 DSNs, 7 CoE 
directors, 3 IDRC directors, and two TAPs). 

All interviews were approximately an hour long, 
followed a semi-structured interview protocol, 
and were conducted by the same interviewer. 

The interview protocols focused on questions 
regarding the grantees’ role, the services they 
provided, their use of data, and their assessment 
of EWD’s impact on economic and workforce 
development. All interviews were transcribed 
and coded to uncover dominant themes. 

We also include open-ended questions in our surveys 
of DSNs, regional consortia chairs, CTE deans, 
and employers (see below for more information on 
the surveys). We coded these responses to reveal 
themes within and across participant groups. 
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Quantitative data and analysis 
To build a more generalizable understanding 
based on our preliminary qualitative research 
findings, we conducted surveys of DSNs, regional 
consortia chairs, CTE deans, and employers. All 
surveys were conducted online, and all invitations 
were sent via email with a link to the survey. 

Surveys of DSNs 

For the DSN survey, we emailed DSNs directly, asking 
for their participation in the online survey. Forty-one of 
the 66 DSNs responded, and 38 completed the entire 
survey. We asked DSNs about their role in EWD, how 
they coordinate with stakeholders, their successes 
and challenges, their use of data, their impact, and 
recommendations for program improvement. 

Surveys of regional consortia chairs 
The survey of regional consortia chairs asked 
about collaboration in their regions, including 
successes, challenges, and supports that enable 
success; EWD’s strengths and weaknesses; 

EWD’s effectiveness; and recommendations for 
EWD improvement. The survey was administered 
online. Participants for the survey were recruited by 
CCCCO staff, who sent email on behalf of Edlnsights 
to solicit the chairs’ participation. Eight of the 1 0 
regional consortia chairs participated in the survey. 

Surveys of CTE deans 

The online survey of CTE deans asked about their 
experiences with the EWD program. We sought to 
understand their priorities and goals for their CTE 
programs, their familiarity with EWD, EWD’s impact on 
their programs, EWD’s strengths and weaknesses, and 
the deans’ recommendations for improving EWD. The 
CCCCO sent a message to an email listserv of CTE 
deans with a link to the survey to recruit participants. 
Twenty-one CTE deans participated in the survey. 

Surveys of employers 
We administered an online survey to employers 
across the state. Local employer groups, trade 
associations, and workforce agencies throughout 


California sent an email to their employer mailing 
lists asking for their participation in our survey. 

Since we did not have access to those mailing lists, 
we do not know how many requests those entities 
sent, nor do we know who our respondents were, 
beyond sector and region. We received 40 responses 
with 33 completed surveys. We targeted employers 
widely to capture a more representative sample of 
them. We surveyed employers who had and had not 
received services from EWD to minimize bias and to 
understand how those who had never heard of EWD 
would view the services the program provides. 

CCCCO Management Information System data 
To estimate use of EWD services and student 
outcomes, we used CCCCO MIS data. One of the 
largest limitations of these data were that the data 
simply did not exist to answer some of our research 
questions. For example, there is no systemic 
participant-level data collected on EWD services. 

To organize the data, we received a list from 
the CCCCO of all courses that mapped to the 
Taxonomy of Program (TOP) codes related to 
EWD and the DWM sectors. The CCCCO reported 
that there were a total of 203 courses. While 
some courses were deemed to span multiple 
sectors, for the purposes of our analysis, we 
only used the primary sector for each course. 

EWD quarterly reports 
We analyzed all 2013-2014 grantee quarterly 
data provided to us by the CCCCO. The 97 EWD 
grantees— SNs (n=10), DSNs (n=66), IDRCs 
(n=12), TAPs (n=7), contract education (n=1) and 
TAP coordinator (n=1)— submitted a basic report 
that provided a snapshot of their grant activities 
and outcomes. This report is a tillable PDF, where 
grantees select boxes for the services they 
provided to colleges and/or businesses and enter 
figures for the number of people served, number 
of trainings conducted, and revenue generated 
for businesses, among other items (see Appendix 
D for a sample of the quarterly report template). 


Aiming to Meet Workforce Needs: 
An Evaluation of the Economic and Workforce Development Program 


61 


The CCCCO provided us an Excel report of the 
grantee activity as aggregate data (yearly) and 
disaggregate quarterly data (six quarters in total, 
because some expenditures occurred late due 
to delays or grant extensions but were a part of 
2013-14 fiscal year funding). While all grantees 
submitted at least one quarterly report, only seven 
grantees submitted reports for all six quarters. For 
the purposes of the evaluation analyses, we used 
yearly data. One of the biggest limitations about 
the data was that we do not have unduplicated 
data, meaning grantees could have served the 
same businesses each quarter, and may have only 
served 3,420 businesses, as opposed to 13,723. 
We dealt with the issue by reporting both yearly 
and quarterly data, but we believe the quarterly 
data better represents the number of businesses, 
students, and employees served by EWD. 

Another caveat is that the data were self-reported. 
The CCCCO did not provide grantees with 
a tracking tool, thus leaving grantees to create 
their own (and it is not clear if all grantees did so). 
The end result is that the accuracy of the data are 
questionable. Grantees could have under- and/or 
over-reported activities and outcomes each quarter. 

Lastly, there are some issues with the report 
itself. First, grantees are only allowed to state if 
they offered a service or not. They could have 
worked with colleges on multiple certificates or 
programs of study, but that information cannot be 
captured, since all grantees can do is select the 
box stating they provided that service. Another 
issue is the grant activity categories have not 
been defined by the CCCCO, so there is potential 
for grantees to assign different meanings to each 
category, thus not providing a true representation 
of EWD activities. For example, how are “Total # of 
Completions (i.e., workshops, training, etc.)” and 
“Total Hours of Performance Improvement Training” 
different? Did grantees know the difference? 

Even with caveats and limitations, we can start 
to understand how the CCCCO and others are 


implementing the Economic Workforce Development 
Program (EWD) and Doing What Matters (DWM). 

The first step in analyzing the data was to run high- 
level descriptive statistics for all grantee activities 
in 201 3-201 4. We totaled all data for the individual 
grantee activity categories created by the CCCCO. 

We only collapsed categories related to the number 
of services to colleges and businesses, which each 
contained eight subcategories (such as developing 
and aligning curriculum to the workforce’s needs, 
providing professional development to faculty and 
conducting needs assessments for businesses). 

Limitations 

This evaluation faced major limitations in estimating 
outcomes and with quantitative data. Since we 
had no comparison group, ability to control for 
selection bias, and access to student-level data, 
we were not able to make any estimations of 
causality or impact. Additionally, in providing figures 
related to program use, services, and outcomes, 
we were forced to make many concessions due 
to the lack of accurate and consistent data. 

As with any study, there were data limitations. 

With regard to the quantitative data, we had 
access to two sources of data: the quarterly 
reports that the grantees completed and the MIS 
data from the CCCCO. Both datasets were used 
with heavy caveats, which we discuss next. 

With the quarterly reports that grantees completed, 
the data provided were self-reported by the grantees, 
and there was not a formal reporting or auditing 
structure in place. These structures would help 
ensure the validity and accuracy of the data. 

With the MIS data, it was impossible to identify who 
was an “EWD student.” To address this issue, we 
used a proxy by mapping key courses related to EWD 
sectors to TOP codes. We only used the courses that 
mapped in our analysis. We received this mapping 
from the CCCCO. In total, the CCCCO reported 
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that there were 203 courses in the CCC system. 
While some courses were deemed to span multiple 
sectors, for the purposes of our analysis we only 
used the primary sector for each course. If a student 
took a single course classified in one of the EWD 
sectors, that student is included in our analyses. 

Estimating the impact of a program is difficult 
under any circumstances. In this instance, to 
quantify any causal impact was impossible. First, 
systemic, participant-level data on recipients of 
EWD services did not exist. Second, we had no 
way to control for selection bias, or that students, 
faculty, staff, deans, grantees, and employers made 
decisions strategically and for reasons that were 
not unobservable to us. Third, we only had access 
to aggregate data and not student level data. As 
such, we could not attempt any type of modeling 
that would allow for causal inferences. Given these 
limitations, we report the data we had access to with 
the emphasis that none of these figures should be 
interpreted as impacts or effects. The closest we can 
get to understanding causality is in our qualitative 
research when asking about perceptions of impact. 
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Appendix C. Questions the CCCCO 
Sought to be Answered by the Evaluation 

Process study 

Role of EWD grantees 

• What are the roles and contributions of each 
of the EWD grantees in the DWM system? 

• What are the nature and quality of services 
provided to colleges, businesses, students, 
and employees by EWD grantees? 

• How are grantees contributing to 
structural and policy changes? 

• What levels of coordination exist between 
DWN grantees (including those funded 
by EWD and other CCCCO grants)? 

• How does this coordination work? 

• What are some barriers to 
effective coordination? 

• What are some of the successful strategies 
used to promote successful coordination? 

• Specifically, how do SNs work with 
DSNs in their industry sectors? 

• What kinds of assistance do SNs provide? 

• What are some challenges in 
successful SN-DSN coordination? 

• What are some promising practices? 

• What assistance or guidance do grantees in 
the field need from the chancellor’s office? 

Coordination with stakeholders 

• To what degree do EWD grantees connect 
to K-12 schools, community colleges, 
industry, and workforce and economic 
development systems and programs 
(e.g., Workforce Investment Boards)? 

• What kind of connections are these, 
and how are they made? 

• What are some of the benefits 
of these connections? 


• What are some challenges and successes 
grantees had in creating and maintaining 
connections with each type of entity? 

• What are some best practices for 
connecting with each type of entity? 

Assessment and use of data 

• How is labor market information produced 
by the Centers of Excellence being used 
by EWD grantees and other stakeholders, 
including employers and colleges? 

• How useful do grantees and stakeholders 
find the labor market information provided? 

• What are some recommendations for 
improvement of labor market information 
from the Centers of Excellence? 

• What potential does this labor market 
information have for future use? 

• How is LaunchBoard being used 
by EWD grantees and others? 

• Who is using these tools? 

• Are (and if so, how are) these tools 
being used to inform decisions by 
EWD grantees, community colleges, 
and the chancellor’s office? 

• What are some challenges users have had 
with using LaunchBoard and the data tools? 

• What are some ways that LaunchBoard 
and the data tools have been successfully 
employed by EWD grantees, community 
colleges, and the chancellor’s office? 

Outcomes study* 

Student and worker outcomes 

• How many students and workers 
have received education or workforce 
training services funded by EWD? 

• What are the program completion outcomes 
of students and workers receiving education 
or workforce training services under EWD? 
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• What are the skills or competencies 
attained by these participants? 

• How many participants complete 
their program of study? 

• How many participants received 
a certificate or degree? 

• What are the educational progression 
outcomes of students and workers 
receiving education or workforce 
training services under EWD? 

• How many students continued their 
educations at a community college? 

• How many participants were transfer-ready? 

• How many transferred to a 
four-year institution? 

• If entering, re-entering, or continuing in 
the labor force, what are the employment 
outcomes for EWD students and workers? 

• How many of these participants 
enter employment? 

• If so, what wages are they earning? 

Employer and business outcomes 

• How many businesses were 
served under EWD? 

• What types of services were provided by 
each grantee (e.g., assessments, professional 
development for workers, etc.)? 

• What are the effects on employers or 
businesses of receiving EWD-funded services? 


• What are the strengths and 
weaknesses of EWD? 

• What are specific recommendations 
for strategies to improve the 
effectiveness of EWD? 

‘Note: We attempted to answer the outcomes 
study questions with quantitative data when 
possible, but much of the information came 
from qualitative data. When using the qualitative 
data, the outcomes study answered questions 
such as, “How many students and workers do 
stakeholders believe have received education 
or workforce training services funded by 
EWD?” and, “What do stakeholders think 
are the program completion outcomes of 
students and workers receiving education or 
workforce training services under EWD?” 


Program assessment 

• How has the EWD (via the DWM framework) 
affected the community colleges, industry, 
and the workforce system in California? 

• Has EWD contributed to system change 
and capacity-building in the California 
Community College system? If so, how? 

• What supports are necessary to encourage 
system change and capacity-building? 
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Appendix D. Sample Quarterly Report 

Sector Navigator and Deputy Sector Navigator Quarterly Progress Report 


I. Summary of activities conducted during the quarter (Each item must be complete in order for the form to certify. If an 
item does not apply this quarter, please respond with "0".) 


1. Type of service provided to colleges and employers (check all that apply) 


a) For Colleges 

b) For business owners/managers: 

— Develop and Align Curriculum (U 1) 

C Develop curriculum for business (U 1) 

— Curriculum alignment with Third Party (U 3) 

~ Alignment of sector with education (U 1) 

— Certificate & Program of Study Development (LI 2 and 4) 

— Assessments (business needs, technology needs, 
manufacturing process capability) 

C Curriculum articulation along a career path (U 5) 

~ Small Business Creation and/or exporting modules (LI 7) 

— Professional Development for Faculty (U 6) 

— Professional Development of workers (U 6) 

~ Connect to businesses and/or industries 

~ Connect with Colleges and education 

— Research and Studies 

□ Research 

[~ None 

C None 

Other. Description: 

Other. Description: 

l 1 

1 1 


Employer/Employee/Student Outcomes 

2. Number of businesses served: 

3. Number of students served: 

4. Number of employees served: 

5. Total number of completions (i.e. 
workshops, training, etc) 

6. Total hours of contract education: 

7. Total hours of performance improvement 
training: 

8. Total hours of credit/non-credit 
instruction: 

9. How many were placed in jobs? 

10. How many participated in work based 

learning (i.e. an apprenticeship, | ~| 

internship, etc.)? 

Quantitative Impact on Businesses 

For each of the following, quantify the impact on the business(es) receiving services under the EWD Program during this 
quarter. 

11. How many people were hired? | ~| 

12. How many employees were retained? | ~| 

13. How much new revenue generated? | ~| 

14. Amount of increase in sales: 

15. Were there new products developed? 

16. Were there new services developed? 


O No O Yes 

O No Q Yes 



66 


Education Insights Center at California State University, Sacramento 




Appendix E. Details on Coordinating 
Efforts around Economic and 
Workforce Development 

In this appendix, we provide greater details 
on the main findings presented in the body 
of the report. Detailed findings on internal 
coordination are presented first; detailed 
findings on external coordination are presented 
second. Within each of these sections, we 
organize our findings by stakeholder. 

Internal coordination 

Coordination between EWD grantees 
was at the heart of the structure of the 
program. In this section, we discuss our 
key findings on internal coordination: 

• SNs and DSNs worked closely 
together to advance their sectors. 

• DSNs in the same sector or region 
had strong coordination. 

• Coordination between SNs and DSNs 
with CoEs varied substantially. 

• IDRCs built coordination around 
a specific project. 

SNs and DSNs worked closely 
together to advance their sectors 
SNs and DSNs are at the core of the EWD 
program, and their coordination is essential to 
reaching the program’s goals efficiently. This 
is reflected in the DSN survey, where DSNs 
reported meeting frequently with their SNs (92 
percent met on a weekly or monthly basis), 
typically via conference calls. Strengths of the 
coordination between SNs and DSNs included: 

• Use by SNs of their statewide 
expertise, information, and resources 
to support and guide DSNs. 

• Hands-on work by SNs and DSNs in 
the provision of EWD services. 

One of the most commonly cited supports 
provided by SNs built on SNs’ statewide focus. 
This statewide focus gave them the ability to 
understand the services and resources offered in 


each region across the state. SNs, then, would 
share information and resources from one region 
with all DSNs statewide. This sharing of resources 
and practices would be done informally and 
formally. DSNs also reported that SNs provided 
DSNs with extra funding to support certain efforts. 

SNs reported being highly involved with 
DSNs in the direct provision of services. 

These services ranged from trainings to 
workshops. A shared focus on EWD metrics 
led SNs to help DSNs provide services. 

Achieving successful coordination between 
SNs and DSNs faced some challenges, 
according to DSNs. DSNs reported that: 

• SNs did not listen to the expertise and 
information that DSNs provided. 

• Not all SNs had enough expertise to 
lead their sectors’ statewide efforts. 

Two challenges to successful SN-DSN coordination 
were apparent. First, DSNs expressed that 
communication and knowledge seemed to 
only flow from SNs to DSNs. DSNs desired 
a forum for SNs to hear from them. Second, 

DSNs expressed concern and disenchantment 
about SNs, saying they were not experts in their 
sectors or only understood or focused on small 
segments of their sectors. These SNs were not 
as able to support DSNs’ efforts because they 
did not understand the specific and unique needs 
of developing the workforce in their sectors. 

DSNs in the same sector or region 
had strong coordination 
DSNs within sectors and DSNs within regions 
reported coordinating frequently. DSNs worked 
with each other to share ideas, brainstorm 
solutions to barriers, and build networks. These 
relationships were coordinated by different entities. 
Within sectors, SNs facilitated coordination 
among DSNs. Within regions, regional consortia 
chairs and the DSNs facilitated coordination. 
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One challenge mentioned was the lack of 
coordination between DSNs from different 
sectors and regions. DSNs expressed an interest 
in learning from DSNs in other sectors and 
regions. DSNs requested having a statewide 
DSN meeting with DSNs from all sectors and 
regions to share effective practices. Such 
a meeting would promote coordination among 
DSNs from different sectors and regions. 

Coordination between SNs/DSNs 
and CoEs varied substantially 
SNs, DSNs, and CoEs reported working 
together in a variety of ways - some with high 
levels of coordination and others with little or 
none. Regardless of the level of coordination 
reported, SNs and DSNs commonly noted 
how busy CoEs were and that this affected 
the willingness of SNs and DSNs to request 
labor market information from CoEs. 

In more coordinated efforts, SNs, DSNs, and 
CoEs worked collaboratively to understand 
their regions and sectors’ labor market needs. 
Examples of these collaborations were of CoEs: 

• Working closely to understand the specific 
labor market data needs of SNs and DSNs. 

• Creating new surveys to gather data 
that did not previously exist. 

• Conducting analyses of existing data to 
best answer labor market questions. 

• Providing labor market information 
as requested by SNs and DSNs. 

• Proactively sharing labor market 
information with SNs and DSNs. 

Those that reported a lack of 
coordination noted these barriers: 

• SNs and DSNs did not having enough 
funding to pay for CoE services. 

• SNs and DSNs reported that their CoEs 
did not reach out to them and that with 
the CoEs being out of sight, the SNs and 
DSNs were effectively out of mind. 


IDRCs built coordination around 
a specific project 

EWD grantees noted that IDRCs’ projects 
were another way for stakeholders to 
collaborate formally. An IDRC project would 
focus on solving a specific issue in workforce 
development and involve bringing together 
relevant stakeholders. However, IDRCs did 
not seem to be central mechanisms for 
coordination, since their scope was limited, 
and grantees only mentioned IDRCs when 
questions were asked specifically about them. 

External coordination 

In this section, we discuss our key findings 
regarding coordination with other DWM entities, 
K-12, workforce development organizations, 
and employers and their associations: 

• Regional consortia served as a 
key coordinating structure for 
regional CTE activities. 

• EWD grantees built on old and 
developed new employer relationships 
to collaborate on workforce training. 

• Braiding of SB 1 070 and AB 1 402 funds 
drove EWD coordination with K-12. 

• The CCCCO both supported and 
unintentionally impeded coordination 
through its management of grants. 

• Successful coordination at community 
colleges hinged on several factors 
beyond the control of grantees. 

• EWD grantees and workforce organizations 
regularly coordinated, but diverging 
performance metrics served as a barrier. 

Regional consortia served as 
a key coordinating structure for 
regional CTE activities 
Regional consortia were consistently mentioned 
as an important and principal structure for 
coordination, which confirms their role as a way 
for regional CTE stakeholders to meet and 
discuss key regional issues. Regional consortia 
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promoted coordination through regular and formal 
meetings. They reported joint marketing efforts, 
serving on WIB committees and on county-wide 
and regional workforce development groups, 
and collaborating on developing and delivering 
programs, workshops, and conferences. They 
also reported working together on grants. 

All regional consortia chairs surveyed reported 
that DSNs, CoEs, and CTE faculty and staff 
were actively involved in their consortia. Most 
also reported that other EWD grantees (such 
as SNs and IDRCs) and K-12 representatives 
were actively involved. Half reported that 
employers and WIBs were actively involved. 

While regional consortia were key structures 
in supporting coordination, it was noted that 
community colleges make the ultimate decision 
about CTE issues. Regional consortia were 
viewed as merely providing guidance and 
not having decision-making ability. Another 
reported barrier to effective coordination was 
that only those who attended meetings had 
their voices heard, and heard only by the 
others in attendance. As such, the coordinating 
reach of the regional consortia was largely 
limited to stakeholders engaged in them. 

EWD grantees built upon old and 
developed new employer relationships 
to coordinate on workforce training 
EWD grantees reported using their existing 
relationships, cold calls, and networking events 
to connect and coordinate with employers and 
employer organizations. They reported coordinating 
with employers and employer organizations to 
identify workforce training needs and develop 
workshops and courses to address those needs. 

Most (92 percent) of DSNs reported their 
relationships with employers were useful, and 
they reported meeting frequently with employer 
groups and businesses (89 percent of DSNs 
surveyed met with them on a weekly or monthly 


basis). In coordinating with employer groups, 
trade associations, and their sectors, in general, 
SNs and DSNs reported depending on their 
existing relationships and networks, cold calling 
sector leaders, attending industry-based events 
and conferences, and participating in informal 
sector-related “meetups.” SNs and DSNs also 
reported creating their own advisory boards 
with employers and other sector stakeholders 
in an effort to make more formal connections. 

SNs and DSNs worked with employer groups 
to identify training needs, then worked with 
them to develop coursework to address these 
needs. They then offered the courses either 
directly to students and workers or to community 
colleges to deliver. The relationships between 
SNs and DSNs sometimes were made more 
formal through the use of Memorandums of 
Understanding and Cooperative Agreements. 

Braiding of Career Technical Education 
Pathways Program and EWD Funds Drove 
Coordination with K-12 

Largely due to Career Technical Education 
Pathways Program (SB 1070; Senate Bill 
1 070, Chapter 433, Statutes of 201 2), funding 
that was braided with EWD, DSNs reported 
coordinating with K-12 schools frequently. 16 
They found SB 1070 meetings to be useful in 
coordinating, but did not find CTE advisory 
board meetings to be as effective. 

DSNs met regularly with K-12 schools; 84 percent 
reported meeting at least monthly. DSNs that 
reported building middle and high school career 
pathways as a part of their role used SB 1 070 
meetings and advisory board meetings as their 
main ways to coordinate with K-12. DSNs reported 
that SB 1070 meetings happened regularly 
statewide and were a key way to connect with 
K-12 and to learn more strategies for building 
and maintaining those connections. Attending 
existing CTE advisory boards that include both 
community colleges and K-12 representatives was 
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reported as less effective, as the DSN role in those 
board meetings was minimal. Adult education 
(Assembly Bill 86, Chapter 48, Statutes 2013) 
meetings were also used to make connections, 
but were rarely mentioned by grantees. 

CCCCO both supported and 
unintentionally impeded coordination 
through its management of grants 
EWD grantees did not report coordinating 
with the CCCCO, however they did note how 
the CCCCO role in managing the EWD grants 
impacted their ability to coordinate with other 
entities. While grantees consistently stated that 
the CCCCO had been responsive and supportive, 
inconsistencies in approvals for grantees’ 
use of funds and inconsistent communication 
regarding the role of grantees plagued their 
ability to effectively coordinate with other 
stakeholders. Grantees reported that they felt 
that policies were, at times, created on the fly and 
that different grant monitors approved certain 
activities while other grant monitors denied 
the same activities. These inconsistencies, 
grantees said, made coordination difficult. 

Successful coordination at community 
colleges hinged on several factors 
beyond the control of grantees 
EWD grantees reported making connections to 
and coordinating with community colleges in 
a variety of ways— including providing professional 
development for faculty, upgrading equipment 
for labs, assisting with revising curriculum, 
and providing labor market information and 
marketing research. SNs and DSNs reported 
making connections with community colleges 
by attending events where community college 
faculty and administrators would be in attendance, 
by cold calling faculty, and through pre-existing 
relationships and networks. Regional consortia 
and the California Community College Association 
of Occupational Education were also reported as 
being important in connecting EWD grantees with 
community college faculty and administrators. 


Grantees found that the success of their 
coordination efforts hinged on several things: 

• Community college stakeholders being 
aware of EWD and understanding their role 
in supporting workforce development. 

• Grantees being hosted at a particular 
community college (this also led to their 
isolation from other colleges in their regions). 

• Smaller regions that made visiting 
each college logistically easy. 

• Colleges that were attuned to 
CTE programs and goals. 

• Colleges with familiar governance 
structures for EWD and CTE 

Awareness of EWD varied 
SNs’ and DSNs’ attempts to connect and 
coordinate with community colleges and 
employers were more successful when college 
stakeholders had already heard of EWD and 
understood the roles of SNs and DSNs. This 
information, they felt, came most effectively from 
the CCCCO, which would build awareness among 
community college administrators who then would 
communicate with their faculty about EWD. 

Host colleges presented challenges to grantees 
The hosting of EWD grantees on community 
college campuses could play a key role in 
supporting coordination between EWD grantees 
and the host campuses. But grantees also 
reported that this relationship led to their isolation 
from the non-host campuses in their regions. 

Some grantees reported having responsibilities 
to their host campuses in addition to their 
role as EWD grantees. These responsibilities 
helped to build the grantees’ connections to 
their hosts, but, they noted, took away from 
their time there to work on EWD projects. 

Large region size prevented adequate coverage 
Coordination between DSNs and the colleges 
in their regions was difficult for DSNs with 
large regions - both those with a large number 
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of colleges in their regions and those with 
geographically large regions. DSNs in large regions 
reported not being able to spend time with every 
college in their region even though they sought to. 

College focus on CTE was inconsistent 
Grantees also reported that they had difficulty 
building connections with colleges that were not 
as attuned to CTE (and, of course, those that did 
not offer programs in that sector). DSNs reported 
a wide range in the level of focus on CTE. CTE 
deans and faculty were found to understand the 
goals and objectives of EWD, however, it was 
not clear to grantees if non-CTE administrators 
understood or “even cared” about EWD. SNs and 
DSNs attempted to overcome this by identifying 
supportive faculty and administrators on campuses 
and having them build bridges to other faculty. 

But they found it difficult to get these supportive 
campus-level stakeholders to the table. 

Variation existed in the structure of 
colleges around CTE governance 
The varied structure of the community colleges 
was another barrier for DSNs to overcome. 

While most EWD sectors were considered CTE, 
some were not. This meant that the courses and 
programs had separate processes for program 
revisions and different requirements (if any) for 
the integration of employers on advisory boards. 
Grantees also noted that some colleges had 
a CTE dean that would oversee CTE programs 
on the campus. Grantees reported that having 
a CTE dean made coordination easier because 
the CTE dean would serve a contact person and 
would also represent the CTE programs in the 
regional consortia. In fact, 94 percent of CTE 
deans surveyed had reported meeting with an 
EWD grantee. Some community colleges had EWD 
deans and directors (distinct from CTE deans). It 
was not clear why some had CTE deans, EWD 
deans and EWD directors, and others did not. 


EWD grantees and workforce organizations 
regularly coordinated, but diverging 
performance metrics served as a barrier 
Due to legal requirements and shared goals, EWD 
grantees coordinated with workforce and economic 
development organizations in a variety of ways. 
However, different performance metrics were 
often a barrier to more effective coordination. 

EWD grantees noted that they were required to 
coordinate with their local Workforce Investment 
Boards (WIBs) and that these WIBs were similarly 
required to work with grantees. As a result, they 
frequently attempted to coordinate. Some of 
these agencies were involved in the SB 1070 
grants, creating a formal connection for DSNs 
also involved in the SB 1070 grants. Some SNs 
and DSNs reported sitting on the local WIB and 
SB 1070 advisory boards. SNs and DSNs said 
they usually made connections with workforce 
and economic develop systems and programs 
by attending their meetings and cold calling. 

SNs and DSNs reported working with these 
organizations in a variety of ways. Given their 
broadly aligned goals, nearly all SNs and DSNs 
reported spending a lot of time with them on 
advisory boards and in meetings. Many SNs 
and DSNs said they put on joint programs 
and training, conducted labor market surveys 
together, and collaborated on SB 1070 activities. 

Specific to Workforce Investment Boards 
(WIBs), SNs and DSNs noted that they faced 
big differences in how collaborative the different 
agencies in their region were. They also indicated 
that it was difficult to get traction with the 
WIBs because their specific outcome metrics 
were different. The SNs and DSNs reported 
that their metrics were more focused on the 
community college momentum points and leading 
indicators, such as retention in a CTE pathway or 
completion of a CTE award. WIBs, on the other 
hand, were focused on entering and maintaining 
employment and earnings after six months. 
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Notes 


1 As measured by receiving a C or better in the course. 

2 For Edlnsights’ analyses and recommendations regarding CTE, please visit 

http://edinsiahtscenter.org/Publications/ctl/CateaorvView/mid/421/cateaorvld/23/Career-and-Technical-Education 

3 SB 1402 took effect January 1, 2013, revising previous legislation authorizing the EWD program. For the full 
text, see http://leainfo.leaislature.ca.aov/faces/billTextClient.xhtml7bill id-201 1201 20SB1 402 

4 For more information on EWD’s priority sectors, refer to http://cccewd.net/industrv proarams.cfm. 

5 For more information about the DWM initiative, see http://doingwhatmatters.cccco.edu 

6 Annual numbers calculated by adding figures reported in the quarterly reports for the 2013-2014 fiscal year. 

7 Courses are classified by their Taxonomy of Programs (TOP) code. At times, courses that were considered CTE were not classified as CTE. 

8 Students who earned a course grade of C or better were considered to have passed the course. 

9 See Appendix B for a description on how we identified EWD courses. Furthermore, only credit courses were included in our analysis. 

10 During this time frame, the community college system experienced severe budget cuts, which could be a reason for the 
FTES’ decline. Furthermore, CTE courses tend to be more expensive for students (i.e. higher equipment and material costs), 
so this also could explain the higher decline in FTES. However, we do not have data to support these conclusions. 

1 1 Chancellor’s office approval is required for associate degrees or certificates of achievement for at least 1 8 semester 
units and is optional for certificates of achievement for 12-18 semester units. Colleges can offer certificates 

for fewer than 12 units, or certificates for 12-18 units that have not been approved by the CCCCO, but such 
certificates cannot be referred to as certificates of achievement or be listed on student transcripts. 

12 Congress mandated that community colleges report outcomes for the Carl D. Perkins Career and Technical Education Act of 2006, 

a federal program that funds states for the improvement of secondary and postsecondary CTE programs. The indicators were established 
“to assess the effectiveness of the state in achieving statewide progress in vocational and technical education, and to optimize the 
return of investment of Federal funds in vocational and technical education activities,” pursuant to section 113(a) of Perkins IV. 

13 Why do colleges host EWD grantees? Colleges host EWD grantees because the EWD funds are from the state’s Proposition 
98 allocation. State practice for Proposition 98 funds has been to directly allocate them to community college districts, 

and has generally excluded programs run directly by state agencies, like the CCCCO. While this distinction can be blurry 
when the state agency provides the service on behalf of the colleges, and there have been exceptions to this, the reality 
is that directly allocating the funds to the colleges ensures that the funds come out of the Proposition 98 share. 

14 The CCCCO provided this 2-page document with suggestions on how to select priority and emergent sectors: 

http://doinawhatmatters.cccco.edU/portals/6/docs/RFA/Selectina%20Prioritv%20and%20Emeraent%20Sectors%201 0041 2.pdf 

15 For Edlnsights’ analyses and recommendations regarding CTE, please visit 

http://edinsiahtscenter.ora/Publications/ctl/CateaorvView/mid/421/cateaorvld/23/Career-and-Technical-Education 

16 Career Technical Education Pathways Program (Chapter 433, Statutes of 2012), or SB 1 070, requires the chancellor of the California 
Community Colleges and the superintendent of public instruction to coordinate on improving career pathways between high schools 
and community colleges. For bill text, see http://leainfo.legislature.ca.gov/faces/billNavClient.xhtml7bill id=201 1201 20SB1 070 



This publication was supported by a subaward agreement with the Butte-Glenn Community College District under Prime Agreement Number 14-181-001 from the 
California Community Colleges Chancellor’s Office. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the author(s) 
and do not necessarily reflect the views of the Butte-Glenn Community College District or those of the California Community Colleges Chancellor’s Office. 
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